May 11, 201511 yr Just a quick question. When I am trying to make a VM in the VM Manager I only have the option for VNC I have a GTX980 installed in my PC and VT-d enabled I have pcie_acs_override=downstream in my sys.cfg Does it have to be added after using XML? ie <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline>
May 11, 201511 yr Author This is the current xml I am working with to get passthrough <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>passthrough</name> <uuid>51493f47-a573-4227-8fb2-4d5578d6efca</uuid> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='5'/> <vcpupin vcpu='6' cpuset='6'/> <vcpupin vcpu='7' cpuset='7'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.2'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source file='/mnt/cache/images/win8/Sub/wineTest/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/images/win8/windows8.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/images/win8/virtio.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:ec:ec:65'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <video> <model type='vmvga' vram='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x046d'/> <product id='0xc52b'/> </source> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> </qemu:commandline> </domain>
May 12, 201511 yr Just a quick question. When I am trying to make a VM in the VM Manager I only have the option for VNC I have a GTX980 installed in my PC and VT-d enabled I have pcie_acs_override=downstream in my sys.cfg Does it have to be added after using XML? ie <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> Hmm, eric told me earlier that if you don't see other devices listed, its because no iommu groups are being detected. Can you double check your bios settings, just to be sure that all virtualization settings are on?
May 12, 201511 yr Author Thats what I was thinking as well Jon, the only thing stopping me checking is: Array Started•Parity-Sync: 13,6 % I double checked I enable it but my mobo can be flakey so maybe some OC tweaks messed it up. VT-d was working on an old AMD card so I know my cpu is capable and I assume (hope) my gpu is.
May 12, 201511 yr Author vt-d is now definitely enabled. My GTX980 is showing up in the device list. When I make a VM and start it the unRaid screen disappears and for a second my TV flashes as if its going to show a picture, but it just goes to 'no signal' When the screen goes 'black' and I forcefully shutdown the VM the screen stays the same. Attached are the 4 templates I have tried and each time I tried them I rebooted my server first: _________________________________________ __________________________________________ __________________________________________ But all gave the same result........black screen. Any suggestions/ideas?
May 12, 201511 yr vt-d is now definitely enabled. My GTX980 is showing up in the device list. When I make a VM and start it the unRaid screen disappears and for a second my TV flashes as if its going to show a picture, but it just goes to 'no signal' When the screen goes 'black' and I forcefully shutdown the VM the screen stays the same. Attached are the 4 templates I have tried and each time I tried them I rebooted my server first: _________________________________________ __________________________________________ __________________________________________ But all gave the same result........black screen. Any suggestions/ideas? What specific brand card? Evga? Asus?
May 12, 201511 yr Author Asus STRIX-GTX 980 http://www.asus.com/ie/Graphics_Cards/STRIXGTX980DC2OC4GD5/
May 12, 201511 yr Asus STRIX-GTX 980 http://www.asus.com/ie/Graphics_Cards/STRIXGTX980DC2OC4GD5/ Are you sure you're supplying enough power to the card? That card requires 300W of power by itself. Add your CPU, fans, storage devices, etc, and you could easily be exceeding the threshold required if you're not supplying enough power. Another user here (expressexcess) had a similar problem, shipped us his card, and from our testing, it worked just fine. I'm inquiring with him on his PSU as well, but haven't heard back yet (to be fair, I just sent him the message about this in the past 15 minutes). Having the card reach a POST screen isn't a solid test. Many GPUs will let you go just fine with under-supplied power until you go to play a game. In our case, this may have an impact when passing through.
May 12, 201511 yr Author Thanks Jon. Yes, I have enough power alright I have an Antec High Current Gamer 900W I have 5 drives and a few fans, so it should be enough. I checked out everything on pcpartpicker.com and it all looked fine. I'll double check all connections again. Could it be something to do with drivers on the card by any chance? I'll be gutted if this card dosent work so I'm willing to try anything now.
May 12, 201511 yr Author Theres an 8 pin and a 6 pin connection. There is a little led over each of them and they are lit. So, I dont think its psu or connection issue
May 12, 201511 yr Theres an 8 pin and a 6 pin connection. There is a little led over each of them and they are lit. So, I dont think its psu or connection issue Ok, a few things you can try: 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. Here's how to do it: Ok, let's try this: 1 - Goto this URL: http://www.techpowerup.com/vgabios/index.php Download the latest vbios for your card (look for the specific model) and place this file somewhere on your cache drive or array. Edit the XML for your VM. If SeaBIOS: Edit the QEMU argument at the bottom for the GPU. Look for this: <qemu:arg value='vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> The host= may be different, that's ok. Just add to it after x-vga=on: ,romfile=/mnt/user/path/to/romfile Example: <qemu:arg value='vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/path/to/romfile'/> For OVMF: Edit the <hostdev> tag for your video card. Example: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x0' function='0x0'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> Add the following after the </source> tag: <rom file='/mnt/user/path/to/file'/> Report back how it goes.
May 13, 201511 yr Author 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. I have the latest installed 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. unRAID Version: unRAID Server Plus, Version 6.0-beta15 Motherboard: ASUSTeK COMPUTER INC. - SABERTOOTH X79 Processor: Intel® Core™ i7-3930K CPU @ 3.20GHz Cache: L1-Cache = 32 kB (max. capacity 32 kB) L2-Cache = 256 kB (max. capacity 256 kB) L3-Cache = 12288 kB (max. capacity 12288 kB) Memory: 32768 MB (max. installable capacity 96 GB) ChannelA = 8192 MB, 1066 MHz ChannelB = 8192 MB, 1066 MHz ChannelC = 8192 MB, 1066 MHz ChannelD = 8192 MB, 1066 MHz Network: bond0: fault-tolerance (active-backup) eth0: 1000Mb/s - Full Duplex Uptime: 0 days, 0 hours, 7 minutes, 2 seconds 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? I had passthrough working previously with an old AMD card. I plugged it in along the GTX but unRAID didn't detect it after the reboot. But the GTX is still displaying. Ill swap it around new reboot. 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. Yes, I am. I didn't have them enabled the first time . They are now 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. I have the latest BIOS 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. I tried this but it never created the vm <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/images/win8/Asus.GTX980.4096.140925.rom'/> EDIT: The above did create a VM but it still did not passthrough successfully. Just the screen flicker for a second and get 'no source' When the VM is successfully being passed through should you still see the VNC symbol in the 'Action' ares on the VM page?
May 13, 201511 yr 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. I have the latest installed 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. unRAID Version: unRAID Server Plus, Version 6.0-beta15 Motherboard: ASUSTeK COMPUTER INC. - SABERTOOTH X79 Processor: Intel® Core™ i7-3930K CPU @ 3.20GHz Cache: L1-Cache = 32 kB (max. capacity 32 kB) L2-Cache = 256 kB (max. capacity 256 kB) L3-Cache = 12288 kB (max. capacity 12288 kB) Memory: 32768 MB (max. installable capacity 96 GB) ChannelA = 8192 MB, 1066 MHz ChannelB = 8192 MB, 1066 MHz ChannelC = 8192 MB, 1066 MHz ChannelD = 8192 MB, 1066 MHz Network: bond0: fault-tolerance (active-backup) eth0: 1000Mb/s - Full Duplex Uptime: 0 days, 0 hours, 7 minutes, 2 seconds 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? I had passthrough working previously with an old AMD card. I plugged it in along the GTX but unRAID didn't detect it after the reboot. But the GTX is still displaying. Ill swap it around new reboot. 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. Yes, I am. I didn't have them enabled the first time . They are now 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. I have the latest BIOS 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. I tried this but it never created the vm <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/images/win8/Asus.GTX980.4096.140925.rom'/> EDIT: The above did create a VM but it still did not passthrough successfully. Just the screen flicker for a second and get 'no source' When the VM is successfully being passed through should you still see the VNC symbol in the 'Action' ares on the VM page? First, to respond to that last question, I'm pretty sure the answer is "yes", but I've been running internal builds for a while for which we do NOT do that. The next VM manager build completely reskins the main VMs page to look more like Docker (using icons and context menus as opposed to the old technicolor rainbow of symbols off to the right). A few more things you could try: 1) disable on-board graphics in your motherboard completely (force it to use the discrete GPU for graphics for the unRAID console). 2) on a fresh reboot of the host, try UEFI first, before ever attempting to start any VMs using SeaBIOS. Chances are, this is an oddity with your mobo + that video card, but can't say for certain. There are folks who have successfully passed through GTX 980s, but on this spreadsheet there are only two records of anyone testing the Sabertooth x79 from Asus, and neither were with a 980. If neither of the above options work, the next step would be to try a different motherboard.
May 13, 201511 yr Watching this topic with interest. As Jon said, this is similar to my issue (main thread: http://lime-technology.com/forum/index.php?topic=39523.0) Interestingly, there aren't many similarities between our hardware configs. Please keep us posted! Greg
Archived
This topic is now archived and is closed to further replies.