Win11 Bare Metall VM - Poor NVME Passthrough Performance - VM Engine (KVM)

November 25, 2025Nov 25

Hi everyone,

I have migrated my bare metal Windows 11 installation to an Unraid VM. While the system boots and runs, I am facing severe performance issues, specifically regarding disk I/O latency.

Despite having high-end hardware, the VM feels sluggish. Right-clicking on the desktop takes ~3 seconds to open the context menu. Games take forever to load. Benchmarks confirm a massive bottleneck in 4K Random Reads.

Hardware:

CPU: AMD Ryzen 9 9950X (Zen 5)
Motherboard: ASUS ROG Crosshair X870E Hero (BIOS updated to latest AGESA)
RAM: [2x48GB DDR5-6000]
GPU: NVIDIA RTX 5080 (Passed through)
NVMe (System): Samsung 9100 PRO Gen 5 OEM (Passed through via PCI controller)
Unraid Version: 7.2.2

The Problem: CrystalDiskMark inside the VM shows:

Seq Read: ~2300 MB/s (Seems capped at PCIe 3.0 speeds? The drive is Gen 5).
4K Random Read (Q1T1): ~6 MB/s (This is drastically low and causing the lag).

VM Configuration:

Machine: pc-q35-9.2
Bios: OVMF TPM
vDisk: None (Manual passthrough of the NVMe controller via "Other PCI Devices")
CPU Pinning: Isolated CCD 1 completely (Cores 8-15 + Threads) for the VM. Emulator pinned to Core 0/16 (CCD 0).

What I have tried so far (Extensive Troubleshooting):

1. BIOS / Host Level:

Global C-States: Disabled
IOMMU / SVM: Enabled
PCIe Link Speed for M.2_1: Tried forcing Gen 5 (resulted in boot loop), currently on Auto/Gen4.
Boot args: kernel /bzimage append pcie_acs_override=downstream,multifunction isolcpus=8-15 amd_pstate=disable iommu=pt initrd=/bzroot
- Note: amd_pstate=disable was necessary to prevent cores from dropping to 600 MHz. Now clocks are stable at 3000-5000 MHz.

2. Windows / VM Level:

MSI Mode: Enabled (High Priority) for GPU, Audio Controller, and Standard NVM Express Controller using MSI Utility v3.
VirtIO: All drivers installed (Balloon, Serial, NetKVM) via virtio-win-gt-x64.msi.
Optimizations:
- Windows Power Plan: High Performance.
- Core Isolation / Memory Integrity: OFF.
- BitLocker: OFF / Fully Decrypted.
- Write Caching: Enabled in Device Manager (Turning off buffer flushing caused boot loops).
Cleanup: Removed old "Ghost" devices (Samsung Magician drivers, old hidden devices) to fix timeouts.

Benchmark Results (CrystalDiskMark inside VM):

The Question: Why is the 4K performance so abysmal (6 MB/s) on a direct PCIe passthrough of a Gen 5 SSD? It feels like the interrupt handling or the PCIe link negotiation is broken. The sequential speed of ~2300 MB/s also suggests the drive is running at PCIe 3.0 x4 speeds instead of 4.0 or 5.0.

Attached is my VM XML. Any help or ideas are greatly appreciated!

XML Config:

XML

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm'>
  <name>Windows 11 Q35</name>
  <uuid>c69ace29-a030-1c38-4a27-ecaf5fe16117</uuid>
  <metadata>
    <vmtemplate xmlns="http://unraid" name="Windows 11" iconold="windows11.png" icon="windows11.png" os="windowstpm" webui="" storage="default"/>
  </metadata>
  <memory unit='KiB'>50331648</memory>
  <currentMemory unit='KiB'>50331648</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>16</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='24'/>
    <vcpupin vcpu='2' cpuset='9'/>
    <vcpupin vcpu='3' cpuset='25'/>
    <vcpupin vcpu='4' cpuset='10'/>
    <vcpupin vcpu='5' cpuset='26'/>
    <vcpupin vcpu='6' cpuset='11'/>
    <vcpupin vcpu='7' cpuset='27'/>
    <vcpupin vcpu='8' cpuset='12'/>
    <vcpupin vcpu='9' cpuset='28'/>
    <vcpupin vcpu='10' cpuset='13'/>
    <vcpupin vcpu='11' cpuset='29'/>
    <vcpupin vcpu='12' cpuset='14'/>
    <vcpupin vcpu='13' cpuset='30'/>
    <vcpupin vcpu='14' cpuset='15'/>
    <vcpupin vcpu='15' cpuset='31'/>
    <emulatorpin cpuset='0,16'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-9.2'>hvm</type>
    <loader readonly='yes' type='pflash' format='raw'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram format='raw'>/etc/libvirt/qemu/nvram/c69ace29-a030-1c38-4a27-ecaf5fe16117_VARS-pure-efi-tpm.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' clusters='1' cores='8' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.285-1.iso'/>
      <target dev='hdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0xf'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:33:d4:1f'/>
      <source bridge='br0'/>
      <model type='virtio-net'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/mnt/user/isos/vbios/rtx5080.rom'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x79' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <watchdog model='itco' action='reset'/>
    <memballoon model='none'/>
  </devices>
</domain>

Edited November 25, 2025Nov 25 by D4zz4

Quote

November 26, 2025Nov 26

I'm having very similar issues and recently posted about it:
https://forums.unraid.net/topic/195319-vm-sluggishness-that-has-been-plaguing-me-forever/

I don't seem to be having the same disk latency issues though. My reads and writes are much faster. I am thinking my latency issues are GPU passthrough related at this point. But just about everything else you've posted is the same for me and I've been dealing with it for a while. I'll be following this thread. Maybe between the two of us we can figure it out.

Quote

November 26, 2025Nov 26

Try this:
Go to your VM config and get the PCIe bus address of your NVMe (ex. 43:00.0). Then put this into the terminal, substituting the bus address, and post the result:
lspci -vv -s <bus_address> | grep -E "LnkCap|LnkSta"

Here is what my NVMe drive shows:

root@Tower:~# lspci -vv -s 43:00.0 | grep -E "LnkCap|LnkSta"
                LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkSta: Speed 16GT/s, Width x4
                LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+

This line in particular:
LnkSta: Speed 16GT/s, Width x4

shows my NVMe has negotiated 16GT/s (PCIe 4.0 speeds), which is correct for my system.
Something like this:
LnkSta: Speed 2.5GT/s (downgraded), Width x4
would show it was downgraded, in this case to PCIe 1.0.

That should at least rule one thing out.

Edited November 26, 2025Nov 26 by bobbintb

Quote

November 28, 2025Nov 28

Author

Hey bobintb. Thanks for your reply.

I will do that this weekend and let you know the results.

On 11/26/2025 at 10:32 PM, bobbintb said:
Try this:
Go to your VM config and get the PCIe bus address of your NVMe (ex. 43:00.0). Then put this into the terminal, substituting the bus address, and post the result:
lspci -vv -s <bus_address> | grep -E "LnkCap|LnkSta"
Here is what my NVMe drive shows:
root@Tower:~# lspci -vv -s 43:00.0 | grep -E "LnkCap|LnkSta"
                LnkCap: Port #0, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkSta: Speed 16GT/s, Width x4
                LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
This line in particular:
LnkSta: Speed 16GT/s, Width x4
shows my NVMe has negotiated 16GT/s (PCIe 4.0 speeds), which is correct for my system.
Something like this:
LnkSta: Speed 2.5GT/s (downgraded), Width x4
would show it was downgraded, in this case to PCIe 1.0.

That should at least rule one thing out.

Quote

Win11 Bare Metall VM - Poor NVME Passthrough Performance

Featured Replies

Join the conversation

Account

Navigation

Search

Configure browser push notifications

Chrome (Android)

Chrome (Desktop)

Safari (iOS 16.4+)

Safari (macOS)

Edge (Android)

Edge (Desktop)

Firefox (Android)

Firefox (Desktop)