[GUIDE] Optimizing Windows VMs in unRaid

[Vetteman]

Love Unraid. It is very stable and setting up shares is a breeze. But I very much dislike it's virtualization. I tried everything suggested here and I get terrible performance for my Windows 11 vm and my Ubuntu vm.  I do not run both vms concurrently.

 

I am retired from the IT industry where I worked with Vmware and Citrix.  Due to lack of drivers, we virtualized our Novell Netware servers.

 

My Unraid server is a i7-4770S with 32 gigs of RAM. I get better performance with VirtualBox installed on my I5-2500 Windows 10 computer with 16 gigs of ram. On VirtualBox I get near metal performance for my Ubuntu vm.

 

I seem to have had better vm performance on an earlier build of Unraid as I only fire up a vm pc once or twice every two months.

 

Unraid really needs to make vm easier to do or offer us a VirtualBox docker. If I had the small hardware I would build a Proxmox box and be done with Unraid's virt manager.

Edited August 16, 2023 by Vetteman

[PicPoc]

Wow !! Thanks !

Works like a charm with Windows 10 !

 

But doesn't work with Windows XP. Is there any mod to add ?

[m00nman]

XP may need HPET. I don't believe it supports anything better than that.

[PicPoc]

With Windows 10 it's impressive how well it works but with Windows XP it doesn't work.
HI don't understand everything, I just followed your example.

Again thanks about Win10 !

Too bad for the setting for Windows XP.

[m00nman]

23 hours ago, PicPoc said:

HI don't understand everything, I just followed your example.

I am saying I don't believe Windows XP supports Hyper-V Enlightenments. So for XP, you would need to enable HPET timer. See below. I striked through what you do NOT need to add for XP, but change the HPET line to "yes"

 

[PicPoc]

Thanks for spending time on my topic.
I just tested this and unfortunately it doesn't isolate as well as with Windows 10.

[Linguafoeda]

My issue has been where just idling Windows 11 VM on the desktop puts me at 50-70% CPU usage (4 cores / 8 threads on a i7-10700T) and launching chrome permanently has me using 100% of CPU. I just updated to 6.12.6 to see if that would help but it hasn't changed much. I have bitlocker turned off, not sure about "core isolation" that someone mentioned was on in Windows 11 but not on in Windows 10 (my hot CPU temps / 100% pinned usage all started happening when i enabled secure boot and TPM on my Windows 10 VM to allow it to upgrade to Win11 a few months ago...Windows 10 was running smooth as butter with low resource usage).

 

i now made the 3 changes from @m00nman original post + changing the network from virtio-net to virtio. I am hopeful this will fix my very slow Windows VM usage (I've noticed it slightly come down when re-launching but still is keeping my CPU pegged pretty high). I don't have multiple VMs running, just this main Win11 one with 8GB dedicated RAM + 4 cores of my 8 core CPU (i usually average ~16GB of total RAM usage on my unraid server inclusive of my VM on so not sure #2 change does anything?).

 

Also - I'm missing an option to sleep the Windows 11 VM completely (the ctrl+X menu only shows signout/shutdown/restart). does anyone know how to enable this / have it automatically sleep after 30-60 mins to free up resources on my unraid server? I also am unable to hibernate the VM itself - it gives error "Domain 'Windows' could not be suspended. internal unable to execute QEMU agent command 'guest-suspend-disk': Guest agent command failed, error was 'suspend-to-disk not supported by OS'."

Edited December 6, 2023 by Linguafoeda

[Linguafoeda]

On 12/5/2023 at 10:39 PM, Linguafoeda said:

My issue has been where just idling Windows 11 VM on the desktop puts me at 50-70% CPU usage (4 cores / 8 threads on a i7-10700T) and launching chrome permanently has me using 100% of CPU. I just updated to 6.12.6 to see if that would help but it hasn't changed much. I have bitlocker turned off, not sure about "core isolation" that someone mentioned was on in Windows 11 but not on in Windows 10 (my hot CPU temps / 100% pinned usage all started happening when i enabled secure boot and TPM on my Windows 10 VM to allow it to upgrade to Win11 a few months ago...Windows 10 was running smooth as butter with low resource usage).

 

i now made the 3 changes from @m00nman original post + changing the network from virtio-net to virtio. I am hopeful this will fix my very slow Windows VM usage (I've noticed it slightly come down when re-launching but still is keeping my CPU pegged pretty high). I don't have multiple VMs running, just this main Win11 one with 8GB dedicated RAM + 4 cores of my 8 core CPU (i usually average ~16GB of total RAM usage on my unraid server inclusive of my VM on so not sure #2 change does anything?).

 

Also - I'm missing an option to sleep the Windows 11 VM completely (the ctrl+X menu only shows signout/shutdown/restart). does anyone know how to enable this / have it automatically sleep after 30-60 mins to free up resources on my unraid server? I also am unable to hibernate the VM itself - it gives error "Domain 'Windows' could not be suspended. internal unable to execute QEMU agent command 'guest-suspend-disk': Guest agent command failed, error was 'suspend-to-disk not supported by OS'."

 

Bump. Still experiencing very laggy VM with CPU usage very high. Any tips on how to improve performance…it was never this bad on win10 vm

[Dase]

On 12/5/2023 at 8:39 PM, Linguafoeda said:

I have bitlocker turned off, not sure about "core isolation" that someone mentioned was on in Windows 11 but not on in Windows 10 (my hot CPU temps / 100% pinned usage all started happening when i enabled secure boot and TPM on my Windows 10 VM to allow it to upgrade to Win11 a few months ago...Windows 10 was running smooth as butter with low resource usage).

 

Just open the Windows Start menu and type "core isolation". It'll take you to that settings page. Make sure "Memory integrity" is disabled. That definitely slowed down my Windows 11 VM.

[Linguafoeda]

4 hours ago, Dase said:

 

Just open the Windows Start menu and type "core isolation". It'll take you to that settings page. Make sure "Memory integrity" is disabled. That definitely slowed down my Windows 11 VM.

 

I typed it in and see two toggles, one for "memory integrity" and one for "Microsoft vulnerable driver blocklist".

 

Should I just turn the first one off or both?

[Dase]

"Memory integrity" Is the option you should try disabling to see if it makes a difference. Leave the driver blocklist enabled.

[Linguafoeda]

20 hours ago, Dase said:

"Memory integrity" Is the option you should try disabling to see if it makes a difference. Leave the driver blocklist enabled.

 

Just did that and restarted, made quite a difference! The most difference yet so far in terms of performance improvement…thank you so much for the advice. It isn’t fully there yet i.e. I still lag around dragging windows around the screen (I’m not passing through a GPU to windows VM if that matters), but the CPU usage at idle or using chrome is significantly less which helps with my unraid CPU temps

[Linguafoeda]

On 1/22/2023 at 12:35 AM, m00nman said:

3. (do NOT do this if you are gaming on the VM or otherwise need a more predictable performance not affected by all the other containers/VMs running on the same machine) We want to disable CPU pinning completely and let the kernel deal with scheduling and distributing load between all the cores on the CPU. Why is CPU pinning not always good? Let's assume you did your best to distribute and pin cores to different VM. For simplicity let's assume we have a 2 core CPU and 4 VMs. We pin core #1 to VM1 and VM3, and core #2 to VM2 and VM4. Now it so happened that VM1 and VM3 started doing something CPU intensive at the same time and they have to share that core #1 between the two of them all while core #2 is doing completely nothing. By letting kernel schedule the load without pinning it will distribute the load between both cores.

 

Let's go back into the VM settings and

Delete the following block

  <cputune>
    .
    .
    .
  </cputune>

 

Make sure that the line

<vcpu placement='static'>MAX_CPU_NUMBER</vcpu>

and

<topology sockets='1' dies='1' cores='MAX_CPU_NUMBER' threads='1'/>

still has the maximum number of cores your VM is allowed to use (obviously MAX_CPU_NUMBER is a number of cores you want to limit this particular VM to, so replace it with a number)

 

I was reading some reddit Windows optimiztion threads and one of them suggested to use CPU pinning in Unraid -> Settings -> CPU Pinning. I realize i made the above changes that i guess disable CPU pinning. If I only have 1 VM really running 100% of the time (~8GB), and a bunch of docker containers that use maybe another 8GB of RAM of my 32GB total, is there any rhyme or reason for using or not using CPU pinning to pin 3-4 cores (of my 8 cores) to Win11 VM to improve performance?

[m00nman]

4 hours ago, Linguafoeda said:

 

I was reading some reddit Windows optimiztion threads and one of them suggested to use CPU pinning in Unraid -> Settings -> CPU Pinning. I realize i made the above changes that i guess disable CPU pinning. If I only have 1 VM really running 100% of the time (~8GB), and a bunch of docker containers that use maybe another 8GB of RAM of my 32GB total, is there any rhyme or reason for using or not using CPU pinning to pin 3-4 cores (of my 8 cores) to Win11 VM to improve performance?

 

Pinning is only necessary when you want to have predictable performance for a VM/container, especially under heavy load, at the expense of pinned cores just idling and doing nothing when the VM pinned to them is doing nothing.  Lettings the host's kernel distribute the load (no pinning) between unused or underutilized cores will give you better efficiency, and possibly better performance for all the VMs/containers at the expense of 0 predictability whether that particular VM that you use for, let's say, playing games will have adequate CPU utilization headroom for playing games (there are ways to assign priorities for VMs, but it's beyond the scope here). So it really depends on your use case. Enterprise solutions almost never use cpu pinning because they want to extract maximum performance for all of the VMs/containers.

[Linguafoeda]

1 minute ago, m00nman said:

 

Pinning is only necessary when you want to have predictable performance for a VM/container, especially under heavy load, at the expense of pinned cores just idling and doing nothing when the VM pinned to them is doing nothing.  Lettings the host's kernel distribute the load (no pinning) between unused or underutilized cores will give you better efficiency, and possibly better performance for all the VMs/containers at the expense of 0 predictability whether that particular VM that you use for, let's say, playing games will have adequate CPU utilization headroom for playing games (there are ways to assign priorities for VMs, but it's beyond the scope here). So it really depends on your use case. Enterprise solutions almost never use cpu pinning because they want to extract maximum performance for all of the VMs/containers.

 

I see. I'm getting heavy video lag while trying to simply stream and watch a video on the VM through clients including Anydesk and the default noVNC. Do you have any advice or tips on how to significantly improve this video performance? I don't have a GPU to passthrough and my Plex container has the --device=/dev/dri parameter which I believe makes it use my Intel 10th gen iGPU for transcoding

[m00nman]

That's a bad combination. Video driver is emulated + VM decodes the video then reencodes it back to send over the internet via remote desktop app. Remote desktop application aren't really designed for streaming videos at full fps, besides game streaming apps like parsec, but then again you are putting a lot if stress on that cpu without a dedicated gpu to decode and then encode a video stream in real time. Can't you stream video directly to your PC?

[Linguafoeda]

On 1/18/2024 at 2:03 AM, m00nman said:

That's a bad combination. Video driver is emulated + VM decodes the video then reencodes it back to send over the internet via remote desktop app. Remote desktop application aren't really designed for streaming videos at full fps, besides game streaming apps like parsec, but then again you are putting a lot if stress on that cpu without a dedicated gpu to decode and then encode a video stream in real time. Can't you stream video directly to your PC?

Unfortunately not - the site gets blocked on my work PC but they don't block VM software so I want to be able to access video streams.
 

So my only option would be to buy a cheap GPU?

[xtrap225]

i have my work pc on a dedicated 1TB NVMe drive passed through windows 11 vm and i was having loads of issues  getting my cpu usage down. it would always eventually creep up. and stay up.  i tried the "Memory integrity"  recommendation change even though i was probably not supposed to.

what ended up working was disabling "Mitigations Settings" in Unraid Settings. so i just wanted to let folks know somewhere, that made a HUGE difference.

[Linguafoeda]

10 hours ago, xtrap225 said:

i have my work pc on a dedicated 1TB NVMe drive passed through windows 11 vm and i was having loads of issues  getting my cpu usage down. it would always eventually creep up. and stay up.  i tried the "Memory integrity"  recommendation change even though i was probably not supposed to.

what ended up working was disabling "Mitigations Settings" in Unraid Settings. so i just wanted to let folks know somewhere, that made a HUGE difference.

 

Where's this setting? what does it do?

[SimonF]

3 hours ago, Linguafoeda said:

 

Where's this setting? what does it do?

  <cpu mode='host-passthrough' check='none' migratable='off'>
    <topology sockets='1' dies='1' cores='1' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>

 

It means certain cpu functions are not turned off to support migration to another host. You have to edit the xml, there will be an option in the next release.

[Linguafoeda]

20 minutes ago, SimonF said:

  <cpu mode='host-passthrough' check='none' migratable='off'>
    <topology sockets='1' dies='1' cores='1' threads='1'/>
    <cache mode='passthrough'/>
  </cpu>

 

It means certain cpu functions are not turned off to support migration to another host. You have to edit the xml, there will be an option in the next release.

 

oh lol - yes i had that setup from the OP post changes. dope.

[xtrap225]

12 hours ago, Linguafoeda said:

 

Where's this setting? what does it do?

its in your unraid 'settings' section. and it turns of Spectre and the like cpu memory mitigations.

[SimonF]

10 hours ago, Linguafoeda said:

 

oh lol - yes i had that setup from the OP post changes. dope.

Sorry misread the question, you need to install the plugin for security mitigations. 

 

 

[Linguafoeda]

On 1/16/2024 at 5:59 PM, Linguafoeda said:

 

I see. I'm getting heavy video lag while trying to simply stream and watch a video on the VM through clients including Anydesk and the default noVNC. Do you have any advice or tips on how to significantly improve this video performance? I don't have a GPU to passthrough and my Plex container has the --device=/dev/dri parameter which I believe makes it use my Intel 10th gen iGPU for transcoding

 

Just wanted to follow up here that I've been able to significantly improve VM performance to the point it almost feels like native Windows. The two things that made the biggest differences (outside this threads OP for helping reduce my CPU usage significantly):

 

1. installing intel GVT-g plugin to enable hardware acceleration for Windows 11 with my Intel 10th Gen iGPU (which was previously allocating the QuickSync feature to Plex via the extra parameter + Intel GPU TOP plugin)
2. Switching from using NoVNC (native right click within browser) or Anydesk to the Microsoft RDC native application. the performance difference between Microsoft RDC client and Anydesk was pretty significant, and i had been using Anydesk for now 4-5 years under the assumption it was one of the smoother remote clients out there.

 

Youtube videos now play perfectly, there's little perceptible lag and while full screen, the VM runs fantastic (with added benefit that it's offloading lot of work to GPU so my CPU temps stay around 50-60 now).

Edited January 26 by Linguafoeda

[sausuke]

@m00nman
Good day, thanks for this tweaks really helps, can I ask how to do this?

 

Quote

(there are ways to assign priorities for VMs, but it's beyond the scope here)