STUCK trying to pass through my GPU

[rtech73]

Good evening and HAPPY NEW YEAR!

 

I haven't built a computer in 8 years, and have NEVER used Linux, so this may seem like a real NOOB question, but so help me I'm stuck!

 

I have a 30TB Plex Server that I'm rebuilding, but in this new build I want to also have it serve as a Steam Machine, streaming games to my 4 Steam Links throughout the house.

 

unRAID was suggested to me, and I figured I would give it a shot.  I have to say, that I'm really enjoying all that this can do, and I know I'm going to find a lot of uses for it beyond what I originally intended.  

 

I've been looking through a LOT of videos, tutorials and this forum, but am having trouble solving one problem that I'm hoping someone here can help me with.

 

First off, the server:

 

Ryzen 7 2700X

ASRock X570 Steel Legend MB

32GB RAM

An ever increasing supply of HD's (as I transfer them from the old server) but right now just two 4TB drives

1TB nvme SSD (Cache drive)

I was told when I put this system together that this CPU does not have integrated graphics, so the onboard HDMI port would be useless and I'd need a dedicated Graphics Card.   Since I'm not doing anything super intense, I got a Gigabyte GeForce GT 1030.

 

Everything has been going GREAT and I got the PLEX server up and running (I have a PLEX pass) and it's already showing up on my Apple TV PLEX apps).   I have my Windows 10VM up and running and all the drivers installed (using VNC graphics).

 

Since I want to stream gaming graphics, I assumed I needed to pass through the GeForce Card.   My MB supports UIMMO (it's late, did I get that acronym right??) and I have it all enabled, and the settings on the VM now allow me to choose my GeForce card.     HERE is where everything falls apart.

 

As SOON as I activate the VM with the GeForce card enabled, the entire VM freezes up, I get a black screen (locally, still have a monitor on it until I get it all working) and the only way to get everything back to normal is to HARD reset the computer (which, oddly enough, causes the Parity sync to start over, unlike just rebooting through the GUI).   It also now seems to use a lot more CPU (I have it set up using 4 of the 8 cores) when it boots, even having changed it back to VNC, but that may be just me looking for issues where there aren't any, it does settle down).

 

How do I get this Windows 10 VM to work with this graphics card so I can use it as a Steam Gaming Server?

 

OH!   When I look at the VM logs, it says something about not being able to "own" (that may not be totally right, I'm going form memory as for the moment I want the Parity sync to complete before I do this again!) IOMMU group 28 (which is wierd  as that is labeled "Non-Essential Instrumentation... Zeppelin/Renoir PCIe Dummy Function) then "Stuck at D3".   That's the end of the VM log, and the system fully crashes.

 

Thanks SO much everyone for any guidance!!   For now I'm just going to start moving my PLEX stuff over to the new system!

 

Again, HNY!!!

 

Ron

[trurl]

7 hours ago, rtech73 said:

I want the Parity sync to complete

Even after the parity sync completes, if you have to do this:

7 hours ago, rtech73 said:

HARD reset the computer

you are going to get a parity check due to unclean shutdown.

 

 

[rtech73]

3 hours ago, trurl said:

Even after the parity sync completes, if you have to do this:

you are going to get a parity check due to unclean shutdown.

 

 

I guess all the more reason to figure this out - this is eventually gonna be a 40 TB Raid, so if it takes all night to do a 4TB raid with NOTHING ON IT, I can’t even imagine how long it would take to do a 40TB raid full of media!  Lol

[itimpi]

36 minutes ago, rtech73 said:

I guess all the more reason to figure this out - this is eventually gonna be a 40 TB Raid, so if it takes all night to do a 4TB raid with NOTHING ON IT, I can’t even imagine how long it would take to do a 40TB raid full of media!  Lol

Just as a point of interest the time for a parity check is determined by the size of the largest parity drive - it has nothing to do with the amount of storage available in the array.

[rtech73]

2 hours ago, itimpi said:

Just as a point of interest the time for a parity check is determined by the size of the largest parity drive - it has nothing to do with the amount of storage available in the array.

OH!   That's good to know!   Could I ask another side question here?   As someone coming from DrivePool on Windows, where parity was accomplished by being sure there is 2 copies of each file on 2 separate physical drives, I'm curious how 30TB of data can be preserved from a single drive failure amongst say 10 drives all on a single drive?   I've ordered a 6TB drive for my parity drive, as I read that the parity drive needs to be as large as your largest array drive, so for future-proofing, I'm going to do that now before I get too far into this.   How does that data preservation work?

[meep]

4 minutes ago, rtech73 said:

OH!   That's good to know!   Could I ask another side question here?   As someone coming from DrivePool on Windows, where parity was accomplished by being sure there is 2 copies of each file on 2 separate physical drives, I'm curious how 30TB of data can be preserved from a single drive failure amongst say 10 drives all on a single drive?   I've ordered a 6TB drive for my parity drive, as I read that the parity drive needs to be as large as your largest array drive, so for future-proofing, I'm going to do that now before I get too far into this.   How does that data preservation work?

So in very basic terms......

The parity drive stores a bit of data that corresponds to the same bit on each data drive. That bit is either on or off (1 or 0)

So, for the first bit on each drive,  parity calculation looks at the bits across all the data drives. Based on the sum of the bits in that first position, the parity bit is calculated.

If one of the drives fails, the parity bit can be used to calculate what the missing bit would have been, and so the data on the missing disk can be preserved.

 

Let's say you have two data drives and parity. The bits in the first position on the data drives are 1 and 0 respectively. Adding these, we get 1, which is the value stored on the parity drive.

Now, let's say drive 1 fails.  UnRaid now sees 

Drive 1: ?

Drive 2: 0

Parity : 1

 

From this information, unRaid can calculate that the missing bit should be 1. (? = 1 - 0)

 

On of the coolest things you will see in UnRaid is that if you have a drive failure, or you remove a drive, the data that was on that drive will still be available to you. (of course you would not run like this as any further failures will cause data loss)

 

As noted, this has nothing to do with the actual data or how full the drives are. Completely empty and completely full drives have the same number of bits. 

 

[rtech73]

5 minutes ago, meep said:

So in very basic terms......

The parity drive stores a bit of data that corresponds to the same bit on each data drive. That bit is either on or off (1 or 0)

So, for the first bit on each drive,  parity calculation looks at the bits across all the data drives. Based on the sum of the bits in that first position, the parity bit is calculated.

If one of the drives fails, the parity bit can be used to calculate what the missing bit would have been, and so the data on the missing disk can be preserved.

 

Let's say you have two data drives and parity. The bits in the first position on the data drives are 1 and 0 respectively. Adding these, we get 1, which is the value stored on the parity drive.

Now, let's say drive 1 fails.  UnRaid now sees 

Drive 1: ?

Drive 2: 0

Parity : 1

 

From this information, unRaid can calculate that the missing bit should be 1. (? = 1 - 0)

 

On of the coolest things you will see in UnRaid is that if you have a drive failure, or you remove a drive, the data that was on that drive will still be available to you. (of course you would not run like this as any further failures will cause data loss)

 

As noted, this has nothing to do with the actual data or how full the drives are. Completely empty and completely full drives have the same number of bits. 

 

That makes sense.   Kind of a mathematical extrapolation of data.   So, this means (thinking out loud here) that after I'm done transfering all this to my unRAID server, my 35TB duplicated pool on my Windows box is going to have a LOT more storage available to me, since only ONE drive is actually being used to "preserve" the data.   That's pretty sweet.

 

So - as long as I can figure out this show stopping GPU pass-through deal, this is already turning out to be a VERY big step up!!!

[itimpi]

There is a write-up on Unraid parity Here

[meep]

So theres a few things here.

 

If you have nothing on your drives, and you're in the middle of testing and setting stuff up, I wouldn't be running the full parity check every time you hard reset. Yes, if you have important data there, but until you're stable, not necessary.

 

Having said that., there's a bunch of stuff that could be the root cause of the issue you have. Without more info, it's hard to tell, but here's some ideas.....

 

Have you checked the IOMMU grouping in unRaid under Tools->System Devices? Your graphics card should be in its own group, like this;

 

 

That's an AMD, but you get the idea. If there's more than just the VGA and Audio Device in a single group, you need to research and fix that. You may need to adjust your system bios and/or configure unRaid ACS override settings inside VM settings.

• If IOMMU looks good, are you passing through BOTH the video and audio devices?
• It sounds like you have a single GPU. In that scenario, you will likely need to also pass through a GPU Bios to the VM.
• You also need to check if your audio device and video device have passed through to the same bus/slot in the VM XML and that they are configured as multifunction devices.
• You might need to change the VM machine type (from i440fx to q35, or vice versa. You might need to change between OVMF and Seabios in VM config (this requires re-creation of the entire VM)
• You might need to boot unRaid itself in legacy rather than UEFII mode. I've found that to be a LOT more compatible for VMs.

 

The above would be for starters. I know it can be overwhelming, but once you figure it all out, you'll be flying!

 

Rather than go into the details of how to do all of this, I will point you to SpaceInvaderOne on youtube. He's the acknowledged guru of unRaid set up and has some really excellent videos on how to set up VMs, configure device passthorugh etc. Check out his VM creation and GPU passthrough videos.

Edited January 1, 2020 by meep
typos

[rtech73]

I've been looking at some of SpaceInvaderOne's stuff.  He's awesome!   I have a coupe of questions then from this thread:   Is there a way I can stop the parity check from happening automatically after a restart (even a forced restart) without disabling the array?   It's just unnecessary resources, since i don't have anything on it yet!   I want to just play with the VM settings to get this to work!    Also - how do I change from UEFI to legacy boot?   Will that cause any issues with my unRAID install that I've already done?

 

Meep and itimpi, thanks so much for jumping in!   I'll go through these and see what I can do (I have a couple of older cards, so I might even be able to do a BIOD dump from my new card if it gets to that).

[meep]

Don’t think you can prevent parity checks, but I may be corrected on this. You can cancel them if underway by going to main > array operation and clicking cancel.

 

you change from uefi to legacy under main > boot device. Click on the ‘flash’ device name, then ‘syslinux config’ tab. (This is not buried at all!) scroll to the bottom and uncheck the ‘permit uefi boot mode’ checkbox.

 

should not affect anything you’ve done. I find my unraid boots faster and has better passthrough compatibility in legacy mode.

Edited January 1, 2020 by meep

[rtech73]

39 minutes ago, meep said:

Don’t think you can prevent parity checks, but I may be corrected on this. You can cancel them if underway by going to main > array operation and clicking cancel.

 

you change from uefi to legacy under main > boot device. Click on the ‘flash’ device name, then ‘syslinux config’ tab. (This is not buried at all!) scroll to the bottom and uncheck the ‘permit uefi boot mode’ checkbox.

 

should not affect anything you’ve done. I find my unraid boots faster and has better passthrough compatibility in legacy mode.

COOL!   Thank you SO much.   I used to be into stuff like this when I was younger - it's nice to be playing with stuff like this again (I have 4 kids, aged 3-13, and it's amazing how they will make it harder to get to your hobbies!).    Have a good night!   I've got enough to go on for a bit now I think!

 

[rtech73]

Actually - I do have ONE more question.   I found another 256MB PCIe graphics card in my box of parts.   Is there any possibility that this will be useful in getting my new card to pass through easier?

[trurl]

3 hours ago, rtech73 said:

only ONE drive is actually being used to "preserve" the data.

It has been explained pretty well already, and I think you probably do understand, but the way I prefer to say it is parity PLUS all remaining drives allows the data for a missing drive to be calculated. That one parity drive doesn't preserve anything by itself.

[rtech73]

17 minutes ago, trurl said:

It has been explained pretty well already, and I think you probably do understand, but the way I prefer to say it is parity PLUS all remaining drives allows the data for a missing drive to be calculated. That one parity drive doesn't preserve anything by itself.

Yep, they pointed me to the "overview" and that explained it pretty well!

 

Now just trying to get this passthrough to work.  Found another very old PCIe graphics card as I read somewhere that having two GPUs can help, but all that does it freeze the thing up as well.   I'll get there!  You guys are awesome!

[meep]

5 minutes ago, rtech73 said:

Yep, they pointed me to the "overview" and that explained it pretty well!

 

Now just trying to get this passthrough to work.  Found another very old PCIe graphics card as I read somewhere that having two GPUs can help, but all that does it freeze the thing up as well.   I'll get there!  You guys are awesome!

Use the older card in your first slot to force unraid to post to it itself. Your better card might then be easier to pass through as you’re not wresting it away from unraid. All the above advice still applies though. Good luck!

[rtech73]

I've tried EVERYTHING above.   ROM, Q35, new VM, booting server in Legacy, etc.  I think the only thing I haven't looked at is the XML whether the card is being passed through to the same slot - but from what I've read, that would just cause errors in Windows, not an INSTANT fail.   So, I'm going to post the most recent Log, and the VM XML in hopes that those might help someone with a better eye toward this than I have yet?

 

HERE is the log:

 

smp 6,sockets=1,cores=6,threads=1 \
-uuid 3ecb7fc0-eee8-ad2f-d5e9-2b2c05370371 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-drive 'file=/mnt/user/domains/Windows 10/vdisk1.img,format=raw,if=none,id=drive-virtio-disk2,cache=writeback' \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk2,id=virtio-disk2,bootindex=1,write-cache=on \
-drive file=/mnt/user/isos/Win10_1909_English_x64.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on \
-device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 \
-drive file=/mnt/user/isos/virtio-win-0.1.171.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on \
-device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ff:ed:14,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=38,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-device vfio-pci,host=0000:0d:00.0,id=hostdev0,x-vga=on,bus=pci.0,addr=0x5,romfile=/mnt/user/isos/Gigabyte.GT1030.2048.170512.rom \
-device vfio-pci,host=0000:0d:00.1,id=hostdev1,bus=pci.0,addr=0x6 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-01-03 02:26:07.416+0000: Domain id=1 is tainted: high-privileges
2020-01-03 02:26:07.416+0000: Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)
2020-01-03T02:26:19.667440Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3
2020-01-03T02:26:19.667603Z qemu-system-x86_64: vfio: Unable to power on device, stuck in D3
2020-01-03T02:26:58.868469Z qemu-system-x86_64: terminating on signal 15 from pid 3415 (/usr/sbin/libvirtd)
2020-01-03 02:27:00.899+0000: shutting down, reason=shutdown

 

HERE is the VM XML:

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 10</name>
  <uuid>3ecb7fc0-eee8-ad2f-d5e9-2b2c05370371</uuid>
  <description>Steam Machine</description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='10'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='12'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='14'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-i440fx-4.1'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='6' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/domains/Windows 10/vdisk1.img'/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/Win10_1909_English_x64.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.171.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:ff:ed:14'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='pci' managed='yes' xvga='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/Gigabyte.GT1030.2048.170512.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0d' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
</domain>
 

 

THANKS GUYS!

 

(If it matters, I have a GeForce 1060 on the way (turns out the 1030 I have won't support the Transcoding I need from PLEX!)

[rtech73]

I did a BIOD upgrade.   The log file is different now:   Doesn't seem to "crash" everything, but still locks up.    

 

Ideas?   (I did go back and re-enable things like the UIMMO after flashing the BIOS to 2.2)

 

-machine pc-i440fx-4.1,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off \
-cpu host,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1fff,hv-vendor-id=none \
-m 8192 \
-overcommit mem-lock=off \
-smp 6,sockets=1,cores=6,threads=1 \
-uuid 3ecb7fc0-eee8-ad2f-d5e9-2b2c05370371 \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-hpet \
-no-shutdown \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x7.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x7 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x7.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x7.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 \
-drive 'file=/mnt/user/domains/Windows 10/vdisk1.img,format=raw,if=none,id=drive-virtio-disk2,cache=writeback' \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk2,id=virtio-disk2,bootindex=1,write-cache=on \
-drive file=/mnt/user/isos/Win10_1909_English_x64.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on \
-device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 \
-drive file=/mnt/user/isos/virtio-win-0.1.171.iso,format=raw,if=none,id=drive-ide0-0-1,readonly=on \
-device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ff:ed:14,bus=pci.0,addr=0x2 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=38,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-device vfio-pci,host=0000:0d:00.0,id=hostdev0,x-vga=on,bus=pci.0,addr=0x5,romfile=/mnt/user/isos/Gigabyte.GT1030.2048.170512.rom \
-device vfio-pci,host=0000:0d:00.1,id=hostdev1,bus=pci.0,addr=0x6 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-01-03 05:16:27.942+0000: Domain id=1 is tainted: high-privileges
2020-01-03 05:16:27.942+0000: Domain id=1 is tainted: host-cpu
char device redirected to /dev/pts/0 (label charserial0)

[meep]

That looks better all right.

 

When you say 'Doesn't seem to "crash" everything, but still locks up. ', you mean that you don't see the 'cant start device' error but the screen remains resolutely blank?

 

Do you know that the VM has actually started, but is just not using the GPU? Have you any way of remoting in to the VM? Did you install something like SplashTop that you can try connecting to? If the VM is running, this would allow you interact with it and progress troubleshooting (you could explore device manager, for example, to determine if the card is there but showing a fault or error code.)

 

I admire your tenacity here! I will note that I spent a good deal of last weekend trying to get a GTX770 passed through, to no avail. It just would not play ball (I have 2x other VMs with RX570 cards working no problem at all. Sometimes, there just seems to be a voodoo combination of hardware, slot configuration and software that just refuses to play nice.

 

Having said that, here are a few further tips that you could try to help troubleshoot.

 

1. Forget pass through for a moment and get your Windows VM set up properly;

-Remove GPU passthrough and VNC in to the VM and ensure you have installed all the QEMU tools, drivers etc.

-Install spalshtop (or equivalent) to allow you remote in later

-Set up windows to auto-login. You don't want your VM sitting at a windows login screen with no GPU to show you what's happening. If you can confidently reboot the VM without a GPU and it logs in to desktop and is ready for a SpalshTop connection, you're in good shape.

 

2. Set up Virt-Manager docker

This is the QEMU manager that offers somewhat more tweaking capabilities than the unRaid VM manager UI. 

While not a recommended configuration, I find the following useful for trouble-shooting

-Set up GPU passthrough as usual in unRaid VM manager

-Nip over to VIRT-Manager and add in a VNC display. This will add two items to the VM. I cant recall the name, but the second device can be configured as QXL, VirtIo or VGA. set this to VGA. (I found this to be more reliable than the other options)

-Start the VM from within the Virt-Manager

 

The effect here is that you can watch the system boot in the Virt-Manager window to see exactly how far its getting. If it gets to desktop, you can poke around via VNC or SplashTop to see how Windows sees your GPU (if at all).

 

Good luck!

 

 

 

[rtech73]

Thanks for the encouragement!   I'm learning fast!   🙂

 

What makes me think the VM is freezing is that the VNC Remote option doesn't appear on the VM menu when I switch from the VNC graphics to the GT1030.   I feel like I've at least gotten things to where it no longer affects the server itself when I try to get the VM running with the 1030.   It *does* start up properly w/o the 1030 passed through, but the machine is pretty much useless as it is SO obvious that it's struggling w/o proper graphics hardware (Steam won't even LAUNCH Portal 2, gives an error directly referring to inadequate graphics hardware).  

 

So, in the meantime, I'm slowly removing (from DrivePool and physically) drives from the old server, to get ready to transfer about 30TB of data to the new server.   I've upgraded my Parity drive to 6TB (that's parity operation is currently running), and getting as many other things going as possible.   Can you tell that on Monday I'll be back to work and trying to get this as far along as possible!?   LOL

 

I appreciate the help!   I'm having a lot of fun with this!   Though my wife is slightly annoyed with my focus on this project!   

 

Have a great day all!

[rtech73]

I'm in the VM, back to VNC drivers.   As SOON as the VM starts, the fans go crazy in my box for a good couple of minutes, then settle down as long as I don't do much with the system.    What I *did* notice is that now when I change the settings so I can boot into the VM, I get the Windows troubleshooting screens.   That tells me that its WINDOWS that's crashing through the boot process, not the VM/container itself.   Am I close on that?   I dl'd Splashtop streamer and client (seems like a good things to have around anyway!) and will see where that gets me.

 

[rtech73]

OK - GTX1060 arrived and installed.   For some reason, I still don't get the VCN Remote option when the 1060 is passed through (and the 1030) still won't pass through) but I CAN get to it through Splashtop.   Window runs SO SO SO slow.   So slow I haven't had the patience to get in and download the latest drivers for the video card.   So I guess I'm making progress.  🙂

[Sleepyme]

1 hour ago, rtech73 said:

OK - GTX1060 arrived and installed.   For some reason, I still don't get the VCN Remote option when the 1060 is passed through (and the 1030) still won't pass through) but I CAN get to it through Splashtop.   Window runs SO SO SO slow.   So slow I haven't had the patience to get in and download the latest drivers for the video card.   So I guess I'm making progress.  🙂

With tour 1060 passed through (and your vbios). When in windows. Check your cpu usuage by logging into unraid from another device. Does one of your cores pegged to 100%. It took me a good month to have mine setup.

[rojarrolla]

Hello to all! now I'm posting here, it sems that rtech either made it or threw Unraid away.

 

I'm  in the same situation as him, I start the VM but the VNC Remote button does not appeared. I followed the video from spaceinvader one of how to pass through the GPU without the second one, and I downloaded from techpowerup the GPU's bios. Still no luck. 

My server is booting in legacy mode (Dell calls it BIOS mode), Immos are in the same gruop for Video and audio, I tryed changing from I440 to Q35 (but not creating the virtual machine again since I had a hard time passing a bare metal hard drive)

So, do you know any update on this?

 

Thanks!

Quick update: I just tried to do it with a Q35 VM and I got the same result, here is part of my .xml:

<rom file='/mnt/user/isos/Vbios/EVGA.GTX1050.2048.170111.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0a' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/>
    </hostdev>

Any ideas?

Edited September 29, 2020 by rojarrolla

[ghost82]

12 hours ago, rojarrolla said:

and I downloaded from techpowerup the GPU's bios

Don't use a third party bios.

Many of these issues were solved by dumping vbios from the gpu directly: you need to find a way to dump the vbios of YOUR gpu.

Then, if it will not work, we will see