Windows 10 1803 GPU Passthrough not working

[UnspokenDrop7]

I upgraded my Windows 10 VM to version 1803 just recently and during the necessary reboot the VM stopped working. It seems to start (at least unRAID seems to think so) but I can not reach it anymore, it doesn't respond to ping or RDP (the Windows firewall is not the problem, it's switched off). If I switch from GPU passthrough to just VNC it works, I can ping and RDP to it again. So I have tried to start over from a fresh install of Windows 10 1803 but it's the same. I have rebooted the unRAID host but no change.

 

Here is the log from one of many attempts to boot up the VM with GPU passthrough, and then doing a "force stop" 20 min later:

 

2018-06-04 13:26:01.476+0000: starting up libvirt version: 4.0.0, qemu version: 2.11.1, hostname: nas-001
LC_ALL=C PATH=/bin:/sbin:/usr/bin:/usr/sbin HOME=/ QEMU_AUDIO_DRV=none /usr/local/sbin/qemu -name guest=DESKTOP-001,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-DESKTOP-001/master-key.aes -machine pc-i440fx-2.11,accel=kvm,usb=off,dump-guest-core=off,mem-merge=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=none -drive file=/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/etc/libvirt/qemu/nvram/e10cd70f-7874-b372-aca3-3291910b1870_VARS-pure-efi.fd,if=pflash,format=raw,unit=1 -m 4096 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid e10cd70f-7874-b372-aca3-3291910b1870 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-DESKTOP-001/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime -no-hpet -no-shutdown -boot strict=on -device ich9-usb-ehc=pci.0,addr=0x5,romfile=/mnt/user/vms/gpu-roms/asus-gtx-1070.rom -device vfio-pci,host=01:00.0,id=hostdev1,bus=pci.0,addr=0x6 -device vfio-pci,host=02:00.1,id=hostdev2,bus=pci.0,addr=0x8 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x9 -msg timestamp=on
2018-06-04 13:26:01.476+0000: Domain id=1 is tainted: high-privileges
2018-06-04 13:26:01.476+0000: Domain id=1 is tainted: host-cpu
2018-06-04T13:26:01.561925Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/0 (label charserial0)
2018-06-04T13:46:27.662074Z qemu-system-x86_64: terminating on signal 15 from pid 10026 (/usr/sbin/libvirtd)
2018-06-04 13:46:29.063+0000: shutting down, reason=destroyed

 

Is there any one else out there having the same experience? And maybe a solution?

 

Thanks,

Kristofer

[unrateable]

just two ideas. try it with the gpu as secondary and vnc primary and then see if you get the hardware drivers installed. then stop the vm , switch the gpu as primary and restart.

another idea would be to make sure to hide the hypervisor in the xml settings and start the vm with gpu passthrough without a bios file....

 

[UnspokenDrop7]

I was able to borrow a monitor from work. Using this I was able to see that the VM actually booted just fine on GPU passthrough. So I logged on and looked up the IP configuration and could see that the network adapter was configured to use DHCP, despite the fact that I had configured it with static settings. So I changed it back to my static settings and then got a notification that there was another adapter using these settings and I was given the option to remove the other adapter's settings and use them for this adapter. Now I was able to ping and connect to the machine using RDP.

 

So for some reason that is not clear to me the machine used one network adapter when I was using the built-in unRAID VNC and another when I used GPU passthrough. It was working all the time it just didn't occur to me that the machine may have a different IP address when using GPU passthrough.

[itimpi]

When you use the unRAID VNC you are connecting to the unRAID IP address which is running the VNC server and allowing you to ‘view’ the virtual screen associated with the VM.   When using RDP you are connecting to the VM’s IP address as the RDP server is running inside the VM.   It is a requirement in such a case therefore that the VM does not have the same IP address as unRAID itself.   The VM will be configured to use a bridged connection (typically br0) to allow it to have it’s own unique IP address.    It sounds as if you may have tried to configure the VM to have the same static IP address as unRAID which is why a conflict was reported?

[UnspokenDrop7]

OK, but this was not my case. I did assigned the machine it's own IP address but not the same as unRAID. I was able to ping and RDP to the machine when using the built-in unRAID VNC. That would not have worked if there was an IP address conflict. I was also able to surf the Internet and access my file shares on my unRAID host which also would not have been possible if there was an IP address conflict.
It seems to me the VM used one NIC when I configured it for VNC (my static IP settings) and another for GPU passthrough (this one was default and used DHCP).
It may be important to know why this happened but for now I'm just glad that it was solved.

 

Thanks for your input!

 

Hopefully this will be useful for someone (maybe myself :)) in the future experience a similar issue.

[itimpi]

32 minutes ago, UnspokenDrop7 said:

OK, but this was not my case. I did assigned the machine it's own IP address but not the same as unRAID. I was able to ping and RDP to the machine when using the built-in unRAID VNC. That would not have worked if there was an IP address conflict. I was also able to surf the Internet and access my file shares on my unRAID host which also would not have been possible if there was an IP address conflict.
It seems to me the VM used one NIC when I configured it for VNC (my static IP settings) and another for GPU passthrough (this one was default and used DHCP).
It may be important to know why this happened but for now I'm just glad that it was solved.

 

Thanks for your input!

 

Hopefully this will be useful for someone (maybe myself :)) in the future experience a similar issue.

Still not quite sure what you were seeing.

 

When you connect via VNC that would be using the IP address of the unRAID server (unless you were running a VNC server inside the VM).    When connecting via RDP  you would be using the IP address of the VM.  

[UnspokenDrop7]

Yes, and I was able to RDP and ping the VM when I used VNC. But when I switched to GPU passthrough I could neither ping or RDP to the machine. This made me believe that it was not booting correctly when using GPU passthrough.

But what I discovered using a monitor connected to the GPU was that it actually was booting just fine. So I signed in to Windows and could see that the VM was using a dynamic IP address (10.220.0.226) assigned from my DHCP despite the fact that I had assigned it a static IP (10.220.0.13, which is not the same as unRAID) earlier when I connected via VNC. So I changed the network adapter again to my static configuration and when I applied the configuration I got a message that there already was another adapter using this configuration (this adapter was not visible) and I got the option to remove the "old" adapter's configuration and use it for this "new" adapter. Now I was able to ping and RPD to my VM when using GPU passthrough.

 

So my conclusions are:

Every time I booted the machine and used VNC it got my static IP settings.

Every time I booted the machine and used GPU passthrough it got dynamic IP settings from my DHCP, since Windows used another adapter.

 

I did not realize this so I tried to ping and RDP to the static IP even when I used GPU passthrough... but that was of course not working.

How this could happened I do not know, it could be nice to know but I'm satisfied that it was solved.

Edited June 9, 2018 by UnspokenDrop7

[itimpi]

You do seem to have done some comprehensive testing?    What I do not understand why those symptoms are occurring in the first place!  With me the network settings in the VM are not affected by how I decide to access the VM.

[UnspokenDrop7]

It has never been like this before, it's really weird.

I have now created a new Windows 10 1803 VM and tested how it behaves with and without GPU passthrough (using my GTX 1060 this time). And it's the same! I have attached how it looks like in the Device Manager, you can clearly see that there is two different network adapters depending on if I use the built-in unRAID VNC or GPU passthrough.

 

Don't mind the warning for my GTX 1060, it's probably because I have not pointed the VM to the correct VBIOS rom file.

 

 

The only changes from earlier installations is that I now install Windows 10 1803 and uses the virtio-win-0.1.141. I will see if I can find some time to do another Windows 10 1803 installation using the previous virtio drivers I used. And if it possible, to find a Windows 10 1709 to install.

Another thing would also be to upgrade unRAID from 6.5.0 to 6.5.2.

[planetwilson]

I had exactly the same symptom with both an nvidia and an amd card. I had two VM configs set up pointing to the same image though and realised they had different MAC/Ethernet addresses set up on the bridge adapter in the VM XML and wondered if that caused Windows to treat them as different adapters?

 

 

 

[Jcloud]

On 6/10/2018 at 3:21 AM, UnspokenDrop7 said:

And if it possible, to find a Windows 10 1709 to install.

I know I'm an internet stranger, but if you can't find a 1709 image I still have one I use for work, and I know I still have the nasty 1603 build. We can figure something out if you need/want the image (like an ISO file and a torrent/magnet for you to pull it; sftp  - or whatever)

[boxer74]

I think I may be having this issue as well. Any fix?

[UnspokenDrop7]

On 6/11/2018 at 8:20 AM, planetwilson said:

I had exactly the same symptom with both an nvidia and an amd card. I had two VM configs set up pointing to the same image though and realised they had different MAC/Ethernet addresses set up on the bridge adapter in the VM XML and wondered if that caused Windows to treat them as different adapters?

 

 

 

I have no idea, sorry.

 

On 6/15/2018 at 1:51 AM, Jcloud said:

I know I'm an internet stranger, but if you can't find a 1709 image I still have one I use for work, and I know I still have the nasty 1603 build. We can figure something out if you need/want the image (like an ISO file and a torrent/magnet for you to pull it; sftp  - or whatever)

Thanks, but I got a 1709 from work.

 

On 8/8/2018 at 7:38 PM, aberg83 said:

I think I may be having this issue as well. Any fix?

No fix or even clue about what the cause could be.

 

I did some tests with 1709 and older win-virtio drivers but problem persists. I have also upgraded to version 6.5.3 of UnRAID but problem persists.

I should mention that I have not kept track of all possible tests that could be done combining different versions of Windows 10, virtio drivers and UnRAID to see if any specific combination would work for me. At the end of the day it was just easier for me to buy some new hardware (MB, RAM, CPU) and using an old PSU and chassis I had.

[boxer74]

It's stable for me for a while and then all of a sudden flaky for a bit. It's ok now.