VIDEO GUIDE***How to pass through an NVIDIA GPU as primary or only gpu in unRAID


SpaceInvaderOne

Recommended Posts

hi guys i having issues i trying to get like 2 windows 10VMs or later  2 win 98 vms to play my old games networked etc

i have a Asus Maxmimus Hero vIII board  and it has onboard video   and i have a 2nd card  in slot one i cant stick in slot too  it doesnt sit with the expansion slots of the case right   and when i did  the card doesnt work  or it will make my computer constantly reboot  never boots up

  move the card  and all i left is vnc and the intel onboard video card  to select from.. so its tempermental...    

and what is a bios dump for?  and cant i just use firmware from the gigabyte site   and i did download the techpowerup  bios for this card  well i picked the first one  there was a bunch of them didnt know which one to choose

 

but i plan to make the 1VM my dedicated everyday computer  and the 2nd VM  be  if i wanted to play win98 games   

so i been watching videos but stuff not working right    any step by step stuff that works 100% of the time..  i tried to follow linus techs tips video for 2 gamerss 1 cpu but they went toooo fast in the setups etc

Link to comment
  • 2 weeks later...
  • 1 month later...

small question does anyone know how to remove the acpi qemu0002 device?

it seams it triggers 43 mode on nvidia card... 

only way to get rid of it that I found is to set pc-i440fx-1.4

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>d6446ff7-1d2a-dfd1-1c21-5f0d819ce30e</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='6'/>
    <vcpupin vcpu='3' cpuset='7'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-1.4'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/d6446ff7-1d2a-dfd1-1c21-5f0d819ce30e_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='off'/>
      <vapic state='off'/>
      <spinlocks state='off'/>
      <vendor_id state='on' value='ab1234567890'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:41:0e:7c'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/gtx970.me.dump.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>

Screenshot_1.png.36199f9c8bc7b9830ba1892055b0ddd8.png

Edited by thor2002ro
Link to comment
  • 2 months later...
  • 3 weeks later...

Thanks for the great tutorials, @SpaceInvaderOne! I have managed to pass through my GTX 1070 to a Windows 10 VM using an edited vbios. However, I have now run into a bizarre problem that I can't seem to fix. The VM starts fine, it boots to login screen where everything works pretty much normally (except the spinner icon). After I log in, the desktop is visible for 2-3 seconds and after that everything goes black. The VM remains responsive, however. The screen does come back momentarily in certain cases:

  • If I type Ctrl-Alt-Del, it briefly (<1sec) flashes the transition screen. The mouse icon changes when I move it over the links even though the screen is black.
  • After Ctrl-Alt-Del and launching the task manager, the desktop is visible for about 1sec with the task manager launched before turning black again.
  • After Ctrl-Alt-Del and clicking the "Switch User" button, the screen comes back showing the login screen and stays visible for as long as I remain on the login screen. All the menus and even sound works on this screen.

VM and Unraid logs don't show any kind of errors.

Does anyone have ideas? I use the recommended Splashtop Streamer.

Edited by sam19
Link to comment

Hello everyone,

 

I've bought a Gigabyte 1650 Mini OC 4Gb.

I've dumped the vbios with GPUZ and it gave me a file of 1022kb.

Is this ok to use or do i have to hex it?

Currently the 1650 is the 2nd gpu but i want to have it as the primary so i can free a pcie slot.

 

I attached the file if anyone wants to use check it or even use it.

 

Gigabyte GeForce® GTX 1650 Mini ITX OC 4G

GVN1650IO4-00-10

TU117.rom

Link to comment
  • 1 month later...

Anyone got this working with RTX? I have:

MSI MEG X570 ACE

Ryzen 3800X

MSI Gaming X Trio RTX 2080 SUPER

 

Looking to passthrough the RTX 2080 Super to VM as a single GPU, have tried it all, modded vbios, Q35 chipset, stubbed the usb controllers and passthrough.

 

If in the second pci slot there's no problem, with a GTX 970 in the primary. But i want to run the super at full x16 speed.

 

With the attached config and steps taken i get a screen output at 800x600 and i am unable to install the drivers in Windows 10 (Also booted outside of Unraid and pre-installed the drivers, but they take no effect in the VM)

 

I am getting the following in the VM log:

2019-07-31T09:48:52.886459Z qemu-system-x86_64: vfio_region_write(0000:2d:00.0:region3+0x14290, 0x67ab0e0d,8) failed: Device or resource busy

Seems that unraid isn't fully releasing the card so far as I can tell, my only thought would be running unraid truely headless with no gfx at all, so it has no ability to interfere with the card, but don't know if that's possible, and of course there is no way to diagnose then if you have no network. Also FYI I am booting the VM directly from the NVMe drive, with no vdisks.

 

I have attached my VM XML to save the length of my post :P

Win10.xml

Link to comment
  • 1 month later...

I just bought an Nvidia GT 720 (specifically this card) to dedicate to a Windows 10 VM to just overall improve the performance of it as a daily general desktop usage.  The overall UX is sluggish, and my research indicates this is likely due to not having a GPU, so I'm testing that theory with this relatively cheap ($60) GPU.

 

I've noticed a lot of discussion almost entirely to GTX based cards.  Does all this discussion apply to my card as well? I don't plan to share it with other VMs just yet, but could see myself wanting to if that's even possible.

 

 

Link to comment
  • 3 weeks later...

After my testings these days I can confirm that the GPU passthrough does not work on x570 boards (my is an Asus Strix E-Gaming). Neither a Nvidia card nor an AMD card. Doesnt matter in which slot the card is...

Here a reference to the thread I started:

 

Unfortunately unRAID is so far not an option for me.

Link to comment

Hey guys,

 

My VM suddenly stoped working, or at least the GPU has.

I receive this error in the log:

 

ErrorWarningSystemArrayLogin


-drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on \
-drive file=/etc/libvirt/qemu/nvram/099f3d54-9e14-3229-33e8-3ea946969e7f_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 \
-m 24576 \
-realtime mlock=off \
-smp 8,sockets=1,cores=4,threads=2 \
-uuid 099f3d54-9e14-3229-33e8-3ea946969e7f \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=27,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-drive file=/mnt/user/domains/Game1/vdisk1.img,format=qcow2,if=none,id=drive-virtio-disk2,cache=writeback \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk2,id=virtio-disk2,bootindex=1,write-cache=on \
-drive file=/mnt/user/domains/_shared/steam2.img,format=raw,if=none,id=drive-virtio-disk3,cache=writeback \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk3,id=virtio-disk3,write-cache=on \
-drive file=/mnt/user/isos/virtio-win-0.1.160-1.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on \
-device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 \
-netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=31 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:54:99:a3,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=32,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device vfio-pci,host=65:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/user/domains/asus.rom \
-device vfio-pci,host=65:00.1,id=hostdev1,bus=pci.0,addr=0x8 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2019-09-28 11:50:19.679+0000: Domain id=6 is tainted: high-privileges
2019-09-28 11:50:19.679+0000: Domain id=6 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2019-09-28T11:50:21.690037Z qemu-system-x86_64: -device vfio-pci,host=65:00.0,id=hostdev0,bus=pci.0,addr=0x6,romfile=/mnt/cache/domains/asus.rom: Failed to mmap 0000:65:00.0 BAR 3. Performance may be slow
2019-09-28T11:50:29.680345Z qemu-system-x86_64: vfio_region_write(0000:65:00.0:region3+0x40a8, 0xbffe101,8) failed: Device or resource busy
2019-09-28T11:50:29.680389Z qemu-system-x86_64: vfio_region_write(0000:65:00.0:region3+0x15000, 0x0,8) failed: Device or resource busy
2019-09-28T11:50:29.680397Z qemu-system-x86_64: vfio_region_write(0000:65:00.0:region3+0x15008, 0x0,8) failed: Device or resource busy
2019-09-28T11:50:29.680407Z qemu-system-x86_64: vfio_region_write(0000:65:00.0:region3+0x15010, 0x0,8) failed: Device or resource busy

Whats wrong there and how to fix it?

Edited by Jaster
Link to comment
  • 4 weeks later...

Hi!

 

Does anyone have a NVIDIA Geforce GT 630 BIOS that works with passthrough as primary GPU?

 

I've been reading the forums for the past 2 days trying to set it up. I've tried techpowerup.com BIOS dumps, various manufacturer roms. All I get is a black screen - no output to the monitor.

The only thing left for me to try is getting another graphics card and dump the BIOS from the uninitialized GT 630.

 

Thanks!

Edited by trenton
additional info
Link to comment
5 hours ago, trenton said:

Does anyone have a NVIDIA Geforce GT 630 BIOS that works with passthrough as primary GPU?

Hi, the best thing to do is to dump your own vbios; search for SpaceInvaderOne's tutorial (that related to the passthrough of a unique gpu): it explains very well how to dump your bios, simple and fast and you are sure the vbios is ok for your card!

 

Edited by ghost82
Link to comment

I got a second card and managed to dump a prepost GPU rom that works. I haven't tried it as primary passthrough because with the 2nd GPU I don't need it anymore. 
I'm attaching the ROM file in case it helps someone else.

 

My card is a Gigabyte NVIDIA GT 530 2GB (probably this one https://www.techpowerup.com/gpu-specs/gigabyte-gt-630-2-gb-rev-3-0.b2415 )

630_nopost_dump.rom

Link to comment
  • 3 weeks later...
  • 11 months later...

I believe it was this guide I followed when I only had a single GPU (Radeon HD 4650) so I could use the GPU for VM's.  It worked fine in that you could see unRAID booting and then during the script it would stop refreshing and be available to the VM's.  I've since added another GPU and use it for my VM's and would like to restore the primary GPU in case I need to do some command line to recover unRAID, reset Docker/VM's, etc.

 

That said, I don't have any exclusions in the Flash config for this device so I'm unsure how to re-activate it.  I followed this video and went to the directory of the device and found there's a file called enable which had a value of 0 whereas my secondary GPU was set to 1 so I modified this value to 1.  There's another file called reset and I entered it hoping it would reset the device without having to restart the server but it didn't seem to work.  I've other things running so I cannot reset at the moment but wonder if there's anything else to do besides what i already did, did I do the right thing, and is there a way to initialize the device without restarting the server? 

Link to comment

I have a new unraid server its a AMD threadripper 2950x with a 980 GPU

 

i had to install the windows VM via VNC  and that all works fine.

image.png.6ce26ef3f4632be4f5354a1a92af2aa6.png

but once i add the videocard in the template and start it. I see it starting and saying booting from hdd. and black screen and when i check the log i see

kernel: vfio-pci 0000:0b:00.0: BAR 3: can't reserve [mem 0xd0000000-0xd1ffffff 64bit pref]

 

anyone an idea how to fix this ?

Edited by KoNeko
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.