QEMU PCIe Root Port Patch


Recommended Posts

On 1/29/2019 at 8:02 AM, billington.mark said:

<qemu:commandline> <qemu:arg value='-global'/> <qemu:arg value='pcie-root-port.speed=8'/> <qemu:arg value='-global'/> <qemu:arg value='pcie-root-port.width=16'/> </qemu:commandline>

 

 

I can report that after updating to 6.7 rc4 and adding that to the xml, there is no change in reported cinebencz fps performance on high Sierra or Mojave. BUT both vm's also reported correctly the PCIe lane width before the patch. 

----

EDIT

 

Above testing was on a GT710, tested on another server running a 1060 and I'm happy to report the following

 

OceanCL before

368.2 max fps

Bandwidth  host to device: 2.79GB/s, Device to host: 3.09GB/s

 

OceanCL after

764.5 max fps

Bandwidth  host to device: 10.56GB/s, Device to host: 11.65GB/s

 

 

Cuda Z before

host to device: 27.9958MiB/s

device to host: 3085.54MiB/s

 

Cuda Z after

host to device: 10.8593GiB/s

device to host: 11.11552GiB/s

 

 

Only a 1-3fps gain in Valley benchmark, but I suspect I might see some large gains on video editing due to higher bandwidth availability.

 

Nice update!

Edited by 1812
Link to comment
1 hour ago, 1812 said:

 

 

I can report that after updating to 6.7 rc4 and adding that to the xml, there is no change in reported cinebencz fps performance on high Sierra or Mojave. BUT both vm's also reported correctly the PCIe lane width before the patch. 

 

But I'm curious how this helps out windows folks.

Trying now. I can say its not really completely about performance but also latency experienced in windows. DPC latency is all over the place. Can confirm patch is successful =0

 

image.png.e391bad6b5d9da821dfb2d9982261cbe.png

Edited by Jerky_san
Link to comment
26 minutes ago, bastl said:

@Jerky_san can you post your xml for reference?

 

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='6' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Gaming-VM-Test</name>
  <uuid>2dbff225-ec92-e4e2-d5d8-fdf4cd84353b</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>33554432</memory>
  <currentMemory unit='KiB'>33554432</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>16</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='36'/>
    <vcpupin vcpu='2' cpuset='5'/>
    <vcpupin vcpu='3' cpuset='37'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='38'/>
    <vcpupin vcpu='6' cpuset='7'/>
    <vcpupin vcpu='7' cpuset='39'/>
    <vcpupin vcpu='8' cpuset='8'/>
    <vcpupin vcpu='9' cpuset='40'/>
    <vcpupin vcpu='10' cpuset='9'/>
    <vcpupin vcpu='11' cpuset='41'/>
    <vcpupin vcpu='12' cpuset='10'/>
    <vcpupin vcpu='13' cpuset='42'/>
    <vcpupin vcpu='14' cpuset='11'/>
    <vcpupin vcpu='15' cpuset='43'/>
    <emulatorpin cpuset='4,8,36,40'/>
  </cputune>
  <numatune>
    <memory mode='strict' nodeset='0,2'/>
  </numatune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/2dbff225-ec92-e4e2-d5d8-fdf4cd84353b_VARS-pure-efi.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
      <reset state='on'/>
      <vendor_id state='on' value='KVM Hv'/>
      <frequencies state='on'/>
    </hyperv>
  </features>
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>EPYC</model>
    <topology sockets='1' cores='8' threads='2'/>
    <feature policy='require' name='topoext'/>
    <feature policy='disable' name='monitor'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='disable' name='svm'/>
    <feature policy='disable' name='x2apic'/>
    <numa>
      <cell id='0' cpus='0-7' memory='16777216' unit='KiB'/>
      <cell id='1' cpus='8-15' memory='16777216' unit='KiB'/>
    </numa>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <alias name='pci.6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <alias name='pci.7'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='8' port='0x1f'/>
      <alias name='pci.8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1c' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='9' port='0x1d'/>
      <alias name='pci.9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:d7:c8:2c'/>
      <source bridge='br0'/>
      <target dev='vnet2'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-6-Gaming-VM-Test/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x41' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/domains/1080ti.rom'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x41' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x3'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x42' slot='0x00' function='0x3'/>
      </source>
      <alias name='hostdev4'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x1867'/>
        <address bus='1' device='8'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x0b05'/>
        <product id='0x1868'/>
        <address bus='1' device='2'/>
      </source>
      <alias name='hostdev6'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
  <qemu:commandline>
    <qemu:arg value='-global'/>
    <qemu:arg value='pcie-root-port.speed=8'/>
    <qemu:arg value='-global'/>
    <qemu:arg value='pcie-root-port.width=16'/>
  </qemu:commandline>
</domain>

 

Link to comment

Thanks @Jerky_san  You basically added the qemu lines at the end. For me in a test VM the Nvidia driver reports the 1050ti as x16 Gen3 now, before only x1.

 

nvidia.png.5d5b84921a030d992f2be41dbff69152.png

 

Another thing i noticed, this is the first VM using the correct memory i have setup. Usually with strict no matter what, it always used a couple MB from the other node. Coincidence? Never saw that before.

<memory mode='strict' nodeset='0'/>

numastat.png.c56f46046fd2974af4baa1b0e16ff511.png

 

 

3 hours ago, limetech said:

Right, our testing didn't show much speed difference but maybe not configuring properly...

Looks like a slight improvment to me. 😂

 

2053056098_aidaGPUcompareRC3RC4.jpg.4301d543aaf68c3c7b49500afdabe839.jpg

 

Couple more tests will follow tomorrow. Thanks for adding that fix 👍

Link to comment
1 hour ago, bastl said:

Thanks @Jerky_san  You basically added the qemu lines at the end. For me in a test VM the Nvidia driver reports the 1050ti as x16 Gen3 now, before only x1.

 

nvidia.png.5d5b84921a030d992f2be41dbff69152.png

 

Another thing i noticed, this is the first VM using the correct memory i have setup. Usually with strict no matter what, it always used a couple MB from the other node. Coincidence? Never saw that before.


<memory mode='strict' nodeset='0'/>

numastat.png.c56f46046fd2974af4baa1b0e16ff511.png

 

 

Looks like a slight improvment to me. 😂

 

2053056098_aidaGPUcompareRC3RC4.jpg.4301d543aaf68c3c7b49500afdabe839.jpg

 

Couple more tests will follow tomorrow. Thanks for adding that fix 👍

Weirdly I didn't originally. For some reason my root tweak for the gpu only isn't there anymore.

 

I forgot I had tweaked my GPU. All you need to do is on the GPU PCI you set the bus to 08 if your using my XML.. up higher in the XML is a pci-root that is set to 08 but and its multi function

Edited by Jerky_san
Link to comment

Thank you thank you thank you thank you !!!!!!

All my latency issues on my OSX Sierra Render VM are gone like the wind. (had huge issues with midi controller devices and driven mouse/screen movements)

Also in pure speed i noticed a few fps increase on some specific Davinci Resolve performance tests.

 

Link to comment

All tests i did so far, on synthetic benchmarks like cinebench or heaven and superposition you can't see that much of a difference. I guess the fact, that the benchmark loads all shaders and textures at the begining is the reason. Testing FarCry5 the performance gain I see is bigger. I guess games that constantly loading stuff will benefit more from that patch. I think that needs a couple more tests

 

Edit: 

I posted some test in another forum related to this.

 

Edited by bastl
Link to comment
<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='7' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>Windows</name>
  <uuid>HIDDEN</uuid>
  <description></description>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="Nvidia.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>16777216</memory>
  <currentMemory unit='KiB'>16777216</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>10</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='7'/>
    <vcpupin vcpu='2' cpuset='8'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='10'/>
    <vcpupin vcpu='5' cpuset='11'/>
    <vcpupin vcpu='6' cpuset='12'/>
    <vcpupin vcpu='7' cpuset='13'/>
    <vcpupin vcpu='8' cpuset='14'/>
    <vcpupin vcpu='9' cpuset='15'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-3.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/HIDDEN_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='10' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source dev='/dev/disk/by-id/ata-SanDisk_SD8SN8U1T001122_161446440614'/>
      <backingStore/>
      <target dev='hdc' bus='sata'/>
      <boot order='1'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source dev='/dev/disk/by-id/ata-ST8000DM0004-1ZC11G_ZKG00C3A'/>
      <backingStore/>
      <target dev='hdd' bus='sata'/>
      <alias name='sata0-0-3'/>
      <address type='drive' controller='0' bus='0' target='0' unit='3'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='sata0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='RANDOM-MAC-ADR'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-7-Windows/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x2f' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x2f' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x30' slot='0x00' function='0x3'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x1b1c'/>
        <product id='0x0a14'/>
        <address bus='5' device='3'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x28de'/>
        <product id='0x1142'/>
        <address bus='5' device='2'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
  <qemu:commandline>
    <qemu:arg value='-global'/>
    <qemu:arg value='pcie-root-port.speed=8'/>
    <qemu:arg value='-global'/>
    <qemu:arg value='pcie-root-port.width=16'/>
  </qemu:commandline>
</domain>

Am I doing something wrong?

 

Still being reported as PCIE0 - I have however never experienced a significant performance drop going virtual in gaming.

 

grafik.png.44684b03ff34f3782cb3a8833ef04e2b.png

Link to comment

@rix Try the following, Get some load on the GPU for example with the render test in GPUZ and run the following comand in unraid.

lspci -s 43:00.0 -vv | grep LnkSta:

Adjust it so it matches your GPU. 43:00.0 is my passed through 1080ti. 8GT/s is what you wanna see for a x16 Gen3 speed.

 

lspci.thumb.png.be01d58c64caa8e1b17efd4de6a2778a.png

 

Mistake that everyone makes is to trust the link speeds GPUZ is reporting. Even if it's reporting x16 the Nvidia system info is the place shows it right. 

gpu.png.70d782eb101de2e9db5b6bff6e402dc0.png

 

Another tool for testing is concBandwidthTest:

 

https://forums.evga.com/PCIE-bandwidth-test-cuda-m1972266.aspx

 

Run it from the comandline inside your VM and report back the values you get.

test.png.03fc428e737d6cf6dbc96f2b8e818d52.png

 

 

  • Like 1
Link to comment

I've just installed RC4 again (having downgraded to isolate an issue (seems to have been memory).  This patch definitely improves my Gaming VM performance on Windows 10.  I found there to be some weird lag between the VM GPU / Mouse Keyboard which was making it nearly impossible to game.  Seems to have gone away now, which is great.

Link to comment
43 minutes ago, Marshalleq said:

I've just installed RC4 again (having downgraded to isolate an issue (seems to have been memory).  This patch definitely improves my Gaming VM performance on Windows 10.  I found there to be some weird lag between the VM GPU / Mouse Keyboard which was making it nearly impossible to game.  Seems to have gone away now, which is great.

Thanks for the report!

  • Like 1
Link to comment

Hey everyone!  Just thought I'd chime in with an update here on this PCIe Root Port Patch, the Q35 machine type, GPU pass through performance, and the future of those things within Unraid.  I actually had an e-mail discussion with Alex Williamson on this subject.  From Alex:

 

Quote

The best I was able to determine was that Pascal and newer NVIDIA cards, when placed into a Q35 configuration will down clock their link from the endpoint to 2.5GT/s when initialized and fail to up-clock under load. AFAIK this behavior is only seen in PCIe topologies, therefore only on Q35.

 

So if you're using an i440fx VM, this patch will do nothing for you.  Furthermore, i440fx VMs don't have the issue of GPUs not up-clocking under load.  I've tested this on both an RTX 2080 Ti and an EVGA GTX 1080 Ti, both work properly under load and do not show any signs of performance bottlenecks.  As such, the logical question I am asking myself is:  why are people using Q35?

 

I'd like to ask the community at large here to answer that question because while there are a few specific situations where Q35 is required, passing through a GPU to a Windows-based VM is not one of them.  And for anyone in this thread reporting a performance increase after this Q35 patch, I would kindly ask you retest using an i440fx based VM and see if you get any different results.  The only specific reason why someone would perhaps want Q35 is for Linux-based gaming VMs that have an AMD-based GPU passed through to them, but even that very niche use-case could be deprecated as I think the newer kernels have fixed i440fx for that case.

 

I think the only use-case so far that makes sense is for Mac-based VMs, which is understandable, but other than that, who is using Q35 and why not i440fx?

 

Feedback is critical here.  We are seriously considering removing the option to select Q35 for Windows-based VMs from VM manager.  If you can provide a use-case for why we shouldn't, please do!!

 

 

  • Upvote 1
Link to comment

As far as I remember, Q35 is needed to get the AMD audio driver to still work after the first reboot after the driver install. 

The workaround I used was to install the windows audio driver, but that only supports 2 channel audio. 

I didn't try this myself as the server was occupied most of the time and then had a breakdown. New hardware don't play nice with the RX480, so had to install a nvidia card. 

Still have one RX480 in the server that I can test with if I get the time tomorrow. 

 

However, I feel it's a little bit too drastic to remove it from being able to select. 

Link to comment
On 2/23/2019 at 6:30 PM, jonp said:

Feedback is critical here.  We are seriously considering removing the option to select Q35 for Windows-based VMs from VM manager.  If you can provide a use-case for why we shouldn't, please do!!

 

@jonp All of my Windows VM's use Q35. I am upgrading my server MB, CPU, and ram to use server grade hardware. In order for me to transfer my VM's, I will need to create a new VM using Q35 to use as a base template, and then point the xml to use the original virtual disk file. For migration purposes, keeping Q35 is about the only way to properly migrate a VM to ensure hardware compatibility with a new server. Also, migrating Windows based VHD's from other sources will become problematic if not impossible to get right. My other thought is that migrating Windows based VM's from q35 to i440fx could invalidate the Windows license, although I don't know if this is a real problem or not.

 

I had no idea that i440fx is superior to q35. Knowing that, I would have used it over q35 when i originally setup my VM's.

Edited by GHunter
  • Like 1
Link to comment

@jonp

Ive been under the impression for a long time that latency and performance improvements in QEMU needed the Q35 machine type to be taken advantage of. 

All development ive seen, and tips to improve performance, all seem to be around using the Q35 machine type. 

At the end of the day, I want to get as close to bare metal performance as possible, thats my aim. Im in no way preaching that we should all move to Q35. 

Now i have my own performance numbers pre and post patch, i'll happily test the i440fx machine type too. 

 

Ive also posted this over in the Level1Tech forum to ask them the same question, seeing as its them who've pushed for the development on the Q35 machine type to get these PCIe fixes in the first place. 

 

As for removing the option in the GUI for Q35 for windows... I think it would be more appropriate to show a warning if Q35 was selected, as apposed to remove the ability to choose it altogether. 

  • Like 1
Link to comment
On 2/23/2019 at 6:24 PM, saarg said:

As far as I remember, Q35 is needed to get the AMD audio driver to still work after the first reboot after the driver install.

If someone can confirm this, it would be a good reason to keep Q35, but I don't know if this is accurate.

 

On 2/24/2019 at 4:17 AM, GHunter said:

For migration purposes, keeping Q35 is about the only way to properly migrate a VM to ensure hardware compatibility with a new server.

Not sure where you heard that, but it's not true.  i440fx works perfectly fine when moving a VM from one set of electronics to another.  Not sure where you got the impression otherwise.

 

On 2/24/2019 at 4:17 AM, GHunter said:

Also, migrating Windows based VHD's from other sources will become problematic if not impossible to get right.

You mean like Hyper-V, VMWare, and VirtualBox?  I've had no problems using i440fx with any of those in the past, albeit I haven't had to do that in a while.

 

On 2/24/2019 at 4:17 AM, GHunter said:

My other thought is that migrating Windows based VM's from q35 to i440fx could invalidate the Windows license, although I don't know if this is a real problem or not.

If you change the motherboard of your computer, Windows wants you to call MS to reactivate the license (if you're not signed in with a Microsoft account).  This applies the same with virtual motherboards like i440fx and Q35.  If you've associated your registered copy of Windows with your Microsoft account, you'll just need to resign in once you change the gear and all is good.

 

10 hours ago, billington.mark said:

Ive been under the impression for a long time that latency and performance improvements in QEMU needed the Q35 machine type to be taken advantage of. 

All development ive seen, and tips to improve performance, all seem to be around using the Q35 machine type. 

Links to relevant articles you've read would help here, but to be fair, there is a lot of misinformation on the web about KVM and QEMU, especially when it relates to GPU pass through.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.