fadynakhla

Hey thanks for this! Been expirementing with different combinations of smbios mode = host, feature policy='disable' name='hypervisor', and kvm disabled. On my setup at least I get much better cpu limited performance without disabling hypervisor. Going from ~120 fps to ~200 fps with 7 x3d cores of my 7950x3d passed through to the vm in 1080p cyberpunk high. Might be different for you, but thought I'd share this in case it helps someone. I'm still able to get past the SM2 EAC vm check.

Hey, performance is great! When gpu limited its within a few percent of bare metal from my testing. In CPU limited scenarios its a bit more of a hit but its also a core configuration that doesn't have a direct competitor. Not sure you can isolate the full v cache ccd, I used some cli tool from within unraid to generate an image of the cpu topology and that basically told me that cores 0 - 7 are the v cache ones but unraid needs core 0 so I isolated cores 1-7 and passed those to the windows VM.

Is it possible to test NVMEs in a zfs pool? I have 2 1tb drives in a striped zfs pool acting as my cache and I would like to benchmark performance. I see this for the first drive and this on the second drive. First message leads me to believe that testing ssds in a zfs pool is not possible but thought I would check

Ok so final update both new VMs (one with gpu one without) are working fine. Unreachability was because I had forgotten to change virbr0 to br0. Still would appreciate any insights into what went wrong with the first VM but everything is working at least.

I was able to fix this by just adding a new vm and using the same vdisk and share mount. Here is the xml for the new working version would love some help figuring out what the issue was. ubuntu_working.xml This is also still with no gpu passthrough I will test that but also on a new vm rather than editting the current one. Edit: Spoke too soon, I can use VNC now but ssh is still not working will continue debugging and report back

I have two VMs in unraid, Windows 11 for gaming and Ubuntu Server for developing. The idea is that typically my gpu is passed through to Windows 11 and if I want to run gpu accelerated dev workloads I can shut down windows, pass the gpu to ubuntu vm and restart it. I had done this in the past (maybe 6 months ago) and it had worked fine although I was using a different gpu back then. Today I attempted to do this and after rebooting the ubuntu VM it was completely unresponsive. It does not connect to my network (cant see it in router, cant ping, cant ssh), it does not stop gracefully I need to force stop in order for it to shutdown. I did force stop it eventually and remove the gpu passthrough to see if it would work but its the same and VNC console simply says "Guest has not initialized the display (yet)". Not entirely sure how to fix this I would appreciate any help. Here is screenshot of vm tab, vm logs, and xml files: Unfortunately I did not save the xml prior to making changes (rookie mistake I know). ubuntu.log ubuntu.xml Edit: Previous gpu was also a 40 series but I had removed all the nvidia drivers from the vm and was planning on reinstalling them with the new gpu passed in. I also didnt make any changes to the xml directly for passing the gpu I used the gui view to switch from vnc to 4080 and back.

Got it thanks! Ill have to just isolate 1-7

FInal update on this, I'm not sure why but after some testing I isolated the issue to one of the usb controllers on my motherboard. I rearanged the usb devices that I dont want to pass through and tested out other usb controllers and am having no issues at all starting and stopping my VM with the controller passed through and any number of devices attached. To anyone experiencing this sort of issue I would recommend trying other usb controllers.

I recently upgraded to a 7950x3d for the increased core count and additional L3 cache when gaming. The chip is split into two CCD's with 8 cores each, only one ccd (cores 0-7) has the stacked L3 cache. I would prefer if unraid used a core from the second ccd and assumed that if I isolated the first 8 cores for VMs unraid would automatically switch to using core 9 but that wasnt the case. I still saw utilization on core 0 with no vms running and also experienced some unresponsiveness/instability. Is there a way to manually change which core unraid uses to something other than 0?

So I think I have figured out what was going, unfortunately looks like the issue is with the usb hub built into my monitor that I am plugging it in to one of the passed through ports. When I unplug it I can pass the usb controller start the vm then plug the monitor's usb hub in after the vm starts to use it. Kinda annoying but i suppose it works

Update: I was able to create and start a fresh windows vm with virtual graphics and then also a fresh vm with graphics card passed through (nothing else), so I tried to boot the existing vm and whole machine crashed again. The share I created and set for syslogs is still empty though oddly Edit: I set the graphics for the existing vm to virtual and removed all passed through devices (except the boot drive) and can start the vm with vnc Edit 2: System wide crash happens again when I pass through the usb controllers and start...

Actually no I havent I figured it was fine as that was working properly with drivers installing automatically and gaming well etc. Also, I dont have the gpu bound to vfio at boot not sure if that has an impact but passes through fine to both ubuntu and windows at least until windows broke Edit: I'll try out new vm with virtual instead

Cool thank you! I'll set this up and then crash the machine and share syslog when I get the chance. Do you think it would be worth wiping the drive with the windows install? I was considering it but since the issue also occured when I tried to spin up a fresh one without that drive at all I'm assuming its something else.

Yeah just hadn't realized I had made any additional gui changes since but may have been adding usb devices or something I dont have syslog but can set it up then try to start the vm, if i use flash or local (share) would the syslog likely contain useful info to look at what caused the crash or do i need to use remote? Not sure if local syslog would be properly written to for a whole system crash Here is the diagnostics file: tower-diagnostics-20230818-1258.zip

Here is the vm xml; it seems that for some reason the few xml edits I made are no longer there: <?xml version='1.0' encoding='UTF-8'?> <domain type='kvm'> <name>Windows 11</name> <uuid>fba0813a-d1a0-5031-957b-d8cbfb6d1ab6</uuid> <description>Gaming Rig</description> <metadata> <vmtemplate xmlns="unraid" name="Windows 11" icon="windows11.png" os="windowstpm"/> </metadata> <memory unit='KiB'>24641536</memory> <currentMemory unit='KiB'>24641536</currentMemory> <memoryBacking> <nosharepages/> </memoryBacking> <vcpu placement='static'>14</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='9'/> <vcpupin vcpu='2' cpuset='2'/> <vcpupin vcpu='3' cpuset='10'/> <vcpupin vcpu='4' cpuset='3'/> <vcpupin vcpu='5' cpuset='11'/> <vcpupin vcpu='6' cpuset='4'/> <vcpupin vcpu='7' cpuset='12'/> <vcpupin vcpu='8' cpuset='5'/> <vcpupin vcpu='9' cpuset='13'/> <vcpupin vcpu='10' cpuset='6'/> <vcpupin vcpu='11' cpuset='14'/> <vcpupin vcpu='12' cpuset='7'/> <vcpupin vcpu='13' cpuset='15'/> </cputune> <os> <type arch='x86_64' machine='pc-i440fx-7.1'>hvm</type> <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader> <nvram>/etc/libvirt/qemu/nvram/fba0813a-d1a0-5031-957b-d8cbfb6d1ab6_VARS-pure-efi-tpm.fd</nvram> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <hyperv mode='custom'> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vendor_id state='on' value='none'/> </hyperv> </features> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='7' threads='2'/> <cache mode='passthrough'/> <feature policy='require' name='topoext'/> </cpu> <clock offset='localtime'> <timer name='hypervclock' present='yes'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/local/sbin/qemu</emulator> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:92:ef:42'/> <source bridge='br0'/> <model type='virtio-net'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <tpm model='tpm-tis'> <backend type='emulator' version='2.0' persistent_state='yes'/> </tpm> <audio id='1' type='none'/> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x11' slot='0x00' function='0x4'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='no'> <source> <vendor id='0x1b1c'/> <product id='0x0c32'/> </source> <address type='usb' bus='0' port='1'/> </hostdev> <memballoon model='none'/> </devices> </domain>