Manticore Posted May 11, 2015 Share Posted May 11, 2015 Just a quick question. When I am trying to make a VM in the VM Manager I only have the option for VNC I have a GTX980 installed in my PC and VT-d enabled I have pcie_acs_override=downstream in my sys.cfg Does it have to be added after using XML? ie <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> Quote Link to comment
Manticore Posted May 11, 2015 Author Share Posted May 11, 2015 This is the current xml I am working with to get passthrough <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>passthrough</name> <uuid>51493f47-a573-4227-8fb2-4d5578d6efca</uuid> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <memoryBacking> <nosharepages/> <locked/> </memoryBacking> <vcpu placement='static'>8</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='1'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='3'/> <vcpupin vcpu='4' cpuset='4'/> <vcpupin vcpu='5' cpuset='5'/> <vcpupin vcpu='6' cpuset='6'/> <vcpupin vcpu='7' cpuset='7'/> </cputune> <os> <type arch='x86_64' machine='pc-q35-2.2'>hvm</type> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-passthrough'> <topology sockets='1' cores='8' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source file='/mnt/cache/images/win8/Sub/wineTest/vdisk1.img'/> <target dev='hdc' bus='virtio'/> <boot order='1'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/images/win8/windows8.iso'/> <target dev='hda' bus='sata'/> <readonly/> <boot order='2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/mnt/user/images/win8/virtio.iso'/> <target dev='hdb' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:ec:ec:65'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <video> <model type='vmvga' vram='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x046d'/> <product id='0xc52b'/> </source> </hostdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/> </qemu:commandline> </domain> Quote Link to comment
jonp Posted May 12, 2015 Share Posted May 12, 2015 Just a quick question. When I am trying to make a VM in the VM Manager I only have the option for VNC I have a GTX980 installed in my PC and VT-d enabled I have pcie_acs_override=downstream in my sys.cfg Does it have to be added after using XML? ie <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=01:00.1,bus=pcie.0'/>/> </qemu:commandline> Hmm, eric told me earlier that if you don't see other devices listed, its because no iommu groups are being detected. Can you double check your bios settings, just to be sure that all virtualization settings are on? Quote Link to comment
Manticore Posted May 12, 2015 Author Share Posted May 12, 2015 Thats what I was thinking as well Jon, the only thing stopping me checking is: Array Started•Parity-Sync: 13,6 % I double checked I enable it but my mobo can be flakey so maybe some OC tweaks messed it up. VT-d was working on an old AMD card so I know my cpu is capable and I assume (hope) my gpu is. Quote Link to comment
Manticore Posted May 12, 2015 Author Share Posted May 12, 2015 vt-d is now definitely enabled. My GTX980 is showing up in the device list. When I make a VM and start it the unRaid screen disappears and for a second my TV flashes as if its going to show a picture, but it just goes to 'no signal' When the screen goes 'black' and I forcefully shutdown the VM the screen stays the same. Attached are the 4 templates I have tried and each time I tried them I rebooted my server first: _________________________________________ __________________________________________ __________________________________________ But all gave the same result........black screen. Any suggestions/ideas? Quote Link to comment
jonp Posted May 12, 2015 Share Posted May 12, 2015 vt-d is now definitely enabled. My GTX980 is showing up in the device list. When I make a VM and start it the unRaid screen disappears and for a second my TV flashes as if its going to show a picture, but it just goes to 'no signal' When the screen goes 'black' and I forcefully shutdown the VM the screen stays the same. Attached are the 4 templates I have tried and each time I tried them I rebooted my server first: _________________________________________ __________________________________________ __________________________________________ But all gave the same result........black screen. Any suggestions/ideas? What specific brand card? Evga? Asus? Quote Link to comment
Manticore Posted May 12, 2015 Author Share Posted May 12, 2015 Asus STRIX-GTX 980 http://www.asus.com/ie/Graphics_Cards/STRIXGTX980DC2OC4GD5/ Quote Link to comment
jonp Posted May 12, 2015 Share Posted May 12, 2015 Asus STRIX-GTX 980 http://www.asus.com/ie/Graphics_Cards/STRIXGTX980DC2OC4GD5/ Are you sure you're supplying enough power to the card? That card requires 300W of power by itself. Add your CPU, fans, storage devices, etc, and you could easily be exceeding the threshold required if you're not supplying enough power. Another user here (expressexcess) had a similar problem, shipped us his card, and from our testing, it worked just fine. I'm inquiring with him on his PSU as well, but haven't heard back yet (to be fair, I just sent him the message about this in the past 15 minutes). Having the card reach a POST screen isn't a solid test. Many GPUs will let you go just fine with under-supplied power until you go to play a game. In our case, this may have an impact when passing through. Quote Link to comment
Manticore Posted May 12, 2015 Author Share Posted May 12, 2015 Thanks Jon. Yes, I have enough power alright I have an Antec High Current Gamer 900W I have 5 drives and a few fans, so it should be enough. I checked out everything on pcpartpicker.com and it all looked fine. I'll double check all connections again. Could it be something to do with drivers on the card by any chance? I'll be gutted if this card dosent work so I'm willing to try anything now. Quote Link to comment
Manticore Posted May 12, 2015 Author Share Posted May 12, 2015 Theres an 8 pin and a 6 pin connection. There is a little led over each of them and they are lit. So, I dont think its psu or connection issue Quote Link to comment
jonp Posted May 12, 2015 Share Posted May 12, 2015 Theres an 8 pin and a 6 pin connection. There is a little led over each of them and they are lit. So, I dont think its psu or connection issue Ok, a few things you can try: 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. Here's how to do it: Ok, let's try this: 1 - Goto this URL: http://www.techpowerup.com/vgabios/index.php Download the latest vbios for your card (look for the specific model) and place this file somewhere on your cache drive or array. Edit the XML for your VM. If SeaBIOS: Edit the QEMU argument at the bottom for the GPU. Look for this: <qemu:arg value='vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/> The host= may be different, that's ok. Just add to it after x-vga=on: ,romfile=/mnt/user/path/to/romfile Example: <qemu:arg value='vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/path/to/romfile'/> For OVMF: Edit the <hostdev> tag for your video card. Example: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x0' function='0x0'/> </source> <alias name='hostdev1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </hostdev> Add the following after the </source> tag: <rom file='/mnt/user/path/to/file'/> Report back how it goes. Quote Link to comment
Manticore Posted May 13, 2015 Author Share Posted May 13, 2015 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. I have the latest installed 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. unRAID Version: unRAID Server Plus, Version 6.0-beta15 Motherboard: ASUSTeK COMPUTER INC. - SABERTOOTH X79 Processor: Intel® Core™ i7-3930K CPU @ 3.20GHz Cache: L1-Cache = 32 kB (max. capacity 32 kB) L2-Cache = 256 kB (max. capacity 256 kB) L3-Cache = 12288 kB (max. capacity 12288 kB) Memory: 32768 MB (max. installable capacity 96 GB) ChannelA = 8192 MB, 1066 MHz ChannelB = 8192 MB, 1066 MHz ChannelC = 8192 MB, 1066 MHz ChannelD = 8192 MB, 1066 MHz Network: bond0: fault-tolerance (active-backup) eth0: 1000Mb/s - Full Duplex Uptime: 0 days, 0 hours, 7 minutes, 2 seconds 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? I had passthrough working previously with an old AMD card. I plugged it in along the GTX but unRAID didn't detect it after the reboot. But the GTX is still displaying. Ill swap it around new reboot. 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. Yes, I am. I didn't have them enabled the first time . They are now 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. I have the latest BIOS 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. I tried this but it never created the vm <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/images/win8/Asus.GTX980.4096.140925.rom'/> EDIT: The above did create a VM but it still did not passthrough successfully. Just the screen flicker for a second and get 'no source' When the VM is successfully being passed through should you still see the VNC symbol in the 'Action' ares on the VM page? Quote Link to comment
jonp Posted May 13, 2015 Share Posted May 13, 2015 1) Are there any BIOS updates for your motherboard? If so, please apply the latest. I have the latest installed 2) Please post your full system specs (motherboard, processor, and all) so I can review the components. unRAID Version: unRAID Server Plus, Version 6.0-beta15 Motherboard: ASUSTeK COMPUTER INC. - SABERTOOTH X79 Processor: Intel® Core™ i7-3930K CPU @ 3.20GHz Cache: L1-Cache = 32 kB (max. capacity 32 kB) L2-Cache = 256 kB (max. capacity 256 kB) L3-Cache = 12288 kB (max. capacity 12288 kB) Memory: 32768 MB (max. installable capacity 96 GB) ChannelA = 8192 MB, 1066 MHz ChannelB = 8192 MB, 1066 MHz ChannelC = 8192 MB, 1066 MHz ChannelD = 8192 MB, 1066 MHz Network: bond0: fault-tolerance (active-backup) eth0: 1000Mb/s - Full Duplex Uptime: 0 days, 0 hours, 7 minutes, 2 seconds 3) Do you have another GPU or another motherboard you can test with to see if that makes a difference? I had passthrough working previously with an old AMD card. I plugged it in along the GTX but unRAID didn't detect it after the reboot. But the GTX is still displaying. Ill swap it around new reboot. 4) Are you 100% positive that you turned on ALL of your virtualization settings in the BIOS? Sometimes there is another setting hidden somewhere else. Yes, I am. I didn't have them enabled the first time . They are now 5) Is there a video BIOS update for your video card available? If so, you may need to install Windows temporarily somewhere to hook up the card and apply the update, but that could also have an impact. I have the latest BIOS 6) Another option would be to try loading the ROM file for the card manually in the XML. This is how I was able to get GPU pass through to work with an ASUS brand R9 290 that I had which hadn't worked before. In fact, I'd probably try this first because it's fairly easy to do. I tried this but it never created the vm <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/mnt/user/images/win8/Asus.GTX980.4096.140925.rom'/> EDIT: The above did create a VM but it still did not passthrough successfully. Just the screen flicker for a second and get 'no source' When the VM is successfully being passed through should you still see the VNC symbol in the 'Action' ares on the VM page? First, to respond to that last question, I'm pretty sure the answer is "yes", but I've been running internal builds for a while for which we do NOT do that. The next VM manager build completely reskins the main VMs page to look more like Docker (using icons and context menus as opposed to the old technicolor rainbow of symbols off to the right). A few more things you could try: 1) disable on-board graphics in your motherboard completely (force it to use the discrete GPU for graphics for the unRAID console). 2) on a fresh reboot of the host, try UEFI first, before ever attempting to start any VMs using SeaBIOS. Chances are, this is an oddity with your mobo + that video card, but can't say for certain. There are folks who have successfully passed through GTX 980s, but on this spreadsheet there are only two records of anyone testing the Sabertooth x79 from Asus, and neither were with a 980. If neither of the above options work, the next step would be to try a different motherboard. Quote Link to comment
expressexcess Posted May 13, 2015 Share Posted May 13, 2015 Watching this topic with interest. As Jon said, this is similar to my issue (main thread: http://lime-technology.com/forum/index.php?topic=39523.0) Interestingly, there aren't many similarities between our hardware configs. Please keep us posted! Greg Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.