Jump to content
dlandon

Performance Improvements in VMs by adjusting CPU pinning and assignment

239 posts in this topic Last Reply

Recommended Posts

edit in GUI and then editing the xml worked, weird, the code looks the same as the one I entered but couldnt update.

 

Gonna run a test in 3DMARK now and see if it performs better now. (Which it should since I use more cores)

 

sorry on the edit i forgot to tell you, you would need to change this line aswell further down the xml (lack of coffee!)

 

<cpu mode='host-passthrough'>
    <topology sockets='1' cores='8' threads='2'/>

 

this correspondes to the number of vcpus used.

 

for example in ours

 

<vcpu placement='static'>16</vcpu>

 

the 16 is  1 socket  8 cores and 2 threads

 

 

Share this post


Link to post

Ok, well the edit in GUI seems to have added that line as well.

 

11694 CPU score now in 3DMARK - which is about 1,5K less than the others I can see that scored higher than me. I assume they are not running a VM.

http://www.3dmark.com/fs/8908247

 

Hmm, is there anyway to boost the score a bit? Or should I be happy with these results?

:)

Share this post


Link to post

 

 

11694 CPU score now in 3DMARK - which is about 1,5K less than the others I can see that scored higher than me. I assume they are not running a VM.

http://www.3dmark.com/fs/8908247

 

Hmm, is there anyway to boost the score a bit? Or should I be happy with these results?

:)

 

I dont think you will get a higher score unless you assign more cores. This would really be pointless for gaming. It would only give you better physics scores on 3d mark. I think that score is fine for the clock speed and architecture of the chip

On my xeon i overclock the bclk to 104 giving me a 4% clock speed increase (cant use multiplier on xeons). i dont know how your 2670 v1 would overclock.

 

Ok, well the edit in GUI seems to have added that line as well.

yes the gui will add that bit when vcpus are changed added etc on gui. Its because we changed the vcpu count in the xml that didnt match the topoly in the xml.

 

Interestingly you can also change the cpu count aswell (not that it should effect performance, but interesting never the less!)

windows would see the following as

 

<cpu mode='host-passthrough'>
    <topology sockets='1' cores='8' threads='2'/>

1 cpu  8 cores  and 2 threads per core  (total vcpu=16)

 

this could be

 

<cpu mode='host-passthrough'>
    <topology sockets='2' cores='4' threads='2'/>

 

2 cpus  4 cores  2 threads  (total vcpu=16)

 

<cpu mode='host-passthrough'>
    <topology sockets='4' cores='2' threads='2'/>

 

4 cpus    2 cores  2 threads  (total vcpu=16) 

 

although windows 7 home is limited to 1 cpu and windows 7 pro limited to 2 cpus.

I think 10 is also limited to 2 physical cpus (although does support upto 128 gigs ram on home and 512 gigs on the pro)

 

Share this post


Link to post

I was using seabios before, but I was told to switch over to ovmf by dlandon. So that is what I am using now. Going to try some gaming tomorrow. But now it is time for bed.

 

Thank you for all the help tonight, and also thank you dlandon.

Share this post


Link to post

Ok sothis is my

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
append isolcpus=11,12,13,14,15,27,28,29,30,31 initrd=/bzroot

 

<domain type='kvm' id='6' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>KP-Ofiice</name>
  <uuid>53662b7a-d29a-63aa-f802-ca9321149187</uuid>
  <description>KP's Main Work Computer</description>
  <metadata>
    <vmtemplate name="Custom" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>51380224</memory>
  <currentMemory unit='KiB'>51380224</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>10</vcpu>
  <cputune>
    <emulatorpin cpuset='0,16'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='10' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/cache/VMs/KP-Ofiice/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:1a:35:d9'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/KP-Ofiice.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=05:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=01.0'/>
  </qemu:commandline>
</domain>

 

Is this correct? Looking at it all the vcpupin lines are missing even though I know I had them in there when I started the VM. VM is booted fine and now looking at XML and they are missing.

 

I wasn't really having an issue with performance but my hope was that this would fix the sound issues I'm having but no luck in that respect.

 

Share this post


Link to post

This is definitely an issue.. if I reboot the VM the

 

 <cputune>
<vcpupin vcpu='11' cpuset='27'/>
    <vcpupin vcpu='12' cpuset='28'/>
    <vcpupin vcpu='13' cpuset='29'/>
    <vcpupin vcpu='14' cpuset='30'/>
    <vcpupin vcpu='15' cpuset='31'/>
    <emulatorpin cpuset='0,16'/>
  </cputune>

 

becomes

 

 <cputune>
    <emulatorpin cpuset='0,16'/>
  </cputune>

 

and while the VM boots it only has one virtual CPU that it is using at 100%. On the initial boot after I manually edit the cputune section the machine boots with 10 cores like it should

Share this post


Link to post

Ok sothis is my

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
append isolcpus=11,12,13,14,15,27,28,29,30,31 initrd=/bzroot

 

<domain type='kvm' id='6' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>KP-Ofiice</name>
  <uuid>53662b7a-d29a-63aa-f802-ca9321149187</uuid>
  <description>KP's Main Work Computer</description>
  <metadata>
    <vmtemplate name="Custom" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>51380224</memory>
  <currentMemory unit='KiB'>51380224</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>10</vcpu>
  <cputune>
    <emulatorpin cpuset='0,16'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='10' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/cache/VMs/KP-Ofiice/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:1a:35:d9'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/KP-Ofiice.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=2,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=05:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=01.0'/>
  </qemu:commandline>
</domain>

 

Is this correct? Looking at it all the vcpupin lines are missing even though I know I had them in there when I started the VM. VM is booted fine and now looking at XML and they are missing.

 

I wasn't really having an issue with performance but my hope was that this would fix the sound issues I'm having but no luck in that respect.

 

Hi in the Syslinux Configuration you have made a mistake

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest
append isolcpus=11,12,13,14,15,27,28,29,30,31 initrd=/bzroot

 

it should be like this i think

 

default /syslinux/menu.c32
menu title Lime Technology
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel /bzimage
  append isolcpus=11,12,13,14,15,27,28,29,30,31 initrd=/bzroot
label unRAID OS Safe Mode (no plugins)
  kernel /bzimage
  append initrd=/bzroot unraidsafemode
label Memtest86+
  kernel /memtest

 

the append isolcpus=11,12,13,14,15,27,28,29,30,31 initrd=/bzroot part should be in the default boot part of the Syslinux Configuration so it is executed on start of server. :)

 

 

Share this post


Link to post

Thank you.

 

I realized after that my VM is Seabios rather than OVMF and I haven't been able to find any way to switch it. Will this still work or do I ave to start over?

Share this post


Link to post

Thank you.

 

I realized after that my VM is Seabios rather than OVMF and I haven't been able to find any way to switch it. Will this still work or do I ave to start over?

You can just start a new template and choose the vdisk of the vm and it should work.

Share this post


Link to post

Thank you.

 

I realized after that my VM is Seabios rather than OVMF and I haven't been able to find any way to switch it. Will this still work or do I ave to start over?

You can just start a new template and choose the vdisk of the vm and it should work.

 

Oh.. ok that is simple enough.

 

I also just figured out that my cputune is way off and I feel kind of stupid for not seeing it right away.

 

Thanks everyone for the help

Share this post


Link to post

Creating an identical template but with OVMF instead of Seabios led to a blank screen. Not sure if that was because of a failed GPU pass-through or if the machine itself was not booting.

 

That said once I made the change to Syslinux gridrunner pointed out and I fixed the really dumb mistake I was making with cputune the  VM boots using Seabios and the sound issue is fixed (it no longer garbles the bass part of audio).

 

Would there be any benefit in continuing to try to get it from Seabios to OVMF?

 

 

Share this post


Link to post

Creating an identical template but with OVMF instead of Seabios led to a blank screen. Not sure if that was because of a failed GPU pass-through or if the machine itself was not booting.

 

That said once I made the change to Syslinux gridrunner pointed out and I fixed the really dumb mistake I was making with cputune the  VM boots using Seabios and the sound issue is fixed (it no longer garbles the bass part of audio).

 

Would there be any benefit in continuing to try to get it from Seabios to OVMF?

 

Unfortunately its not possible to use the same disk image for both seabios and omvf, it does require a separate install.

I have a win 10 seabios and an omvf. The omvf get slightly higher 3dmark scores and passmark scores, but to be honest in real life situations i cant notice a difference. But me being me i use the omvf almost exclusively because i know its benchmarks were a bit higher !! :)

 

Share this post


Link to post

Thank you. I'll use OMVF in the future but for this VM it is just used for general productivity applications so doesn't need every ounce of performance. The main issue was that pre-pinning / assignment the sound card didn't work properly and that is no longer the case so I am happy.

 

Only thing left to do is get the USB card to pass-through after reboots. Currently it only works after a complete unRaid reboot.

Share this post


Link to post

Hey All!

 

I was wondering if there are any audio engineers out there using this method? I am receiving sound issues when going through my USB DAC. I am going through a single USB port to a hub. I don't know if I am doing it correctly. Also, is there anyway to tell if I appended the string to intrid correctly?

 

Here is my XML:

<domain type='kvm' id='14'>
  <name>Windows 10</name>
  <uuid>dd3e87d6-970b-3a66-af1f-0c4bfe93fa94</uuid>
  <metadata>
    <vmtemplate name="Custom" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='18'/>
    <vcpupin vcpu='2' cpuset='20'/>
    <vcpupin vcpu='3' cpuset='17'/>
    <vcpupin vcpu='4' cpuset='19'/>
    <vcpupin vcpu='5' cpuset='21'/>
    <emulatorpin cpuset='0-1'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
    <loader type='pflash'>/usr/share/qemu/ovmf-x64/OVMF-pure-efi.fd</loader>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/VMs/WIN10/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/Windows10.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/VirtIO/virtio-LATEST-0.1.118.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:06:b1:0d'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

Thanks in advance for your help!

Share this post


Link to post

Hey All!

 

I was wondering if there are any audio engineers out there using this method? I am receiving sound issues when going through my USB DAC. I am going through a single USB port to a hub. I don't know if I am doing it correctly. Also, is there anyway to tell if I appended the string to intrid correctly?

 

Here is my XML:

<domain type='kvm' id='14'>
  <name>Windows 10</name>
  <uuid>dd3e87d6-970b-3a66-af1f-0c4bfe93fa94</uuid>
  <metadata>
    <vmtemplate name="Custom" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='18'/>
    <vcpupin vcpu='2' cpuset='20'/>
    <vcpupin vcpu='3' cpuset='17'/>
    <vcpupin vcpu='4' cpuset='19'/>
    <vcpupin vcpu='5' cpuset='21'/>
    <emulatorpin cpuset='0-1'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
    <loader type='pflash'>/usr/share/qemu/ovmf-x64/OVMF-pure-efi.fd</loader>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/VMs/WIN10/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/Windows10.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/VirtIO/virtio-LATEST-0.1.118.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:06:b1:0d'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

Thanks in advance for your help!

You need to pass through a USB controller for sound to work  correctly when using USB sound cards/DAC. All the sound cards/DAC have had the same problem. You might want to check out archdrafts guide on how to pass through a whole controller here

Share this post


Link to post

Hey All!

 

I was wondering if there are any audio engineers out there using this method? I am receiving sound issues when going through my USB DAC. I am going through a single USB port to a hub. I don't know if I am doing it correctly. Also, is there anyway to tell if I appended the string to intrid correctly?

 

Here is my XML:

<domain type='kvm' id='14'>
  <name>Windows 10</name>
  <uuid>dd3e87d6-970b-3a66-af1f-0c4bfe93fa94</uuid>
  <metadata>
    <vmtemplate name="Custom" icon="windows.png" os="windows"/>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <memoryBacking>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='18'/>
    <vcpupin vcpu='2' cpuset='20'/>
    <vcpupin vcpu='3' cpuset='17'/>
    <vcpupin vcpu='4' cpuset='19'/>
    <vcpupin vcpu='5' cpuset='21'/>
    <emulatorpin cpuset='0-1'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
    <loader type='pflash'>/usr/share/qemu/ovmf-x64/OVMF-pure-efi.fd</loader>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/user/VMs/WIN10/vdisk1.img'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/Windows10.iso'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <boot order='2'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/ISOs/VirtIO/virtio-LATEST-0.1.118.iso'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:06:b1:0d'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </memballoon>
  </devices>
</domain>

 

Thanks in advance for your help!

You need to pass through a USB controller for sound to work  correctly when using USB sound cards/DAC. All the sound cards/DAC have had the same problem. You might want to check out archdrafts guide on how to pass through a whole controller here

 

Hey Saarg!

 

I do currently have my USB passthrough functioning with this. it passes through my whole controller. for some reason the qemu from archedraft wasn't functioning for me. It would always revert back to this style. I just knew the address of the PCI for my usb and copy pasted the hostdev from my GPU assignment, and gave it its own slot

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>

 

I'm just not sure if the CPUs are pinned correctly. OP said that we shouldn't change because it would change by itself

<cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>

 

but I changed mine to

 
<cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>

because I am using 3 CPUs and 6 threads (I think thats right)

 

I messed around with the buffer of my DAC a little bit but still getting some underruns and some clipping.

 

still tweaking about...

Share this post


Link to post

Hey All!

 

I was wondering if there are any audio engineers out there using this method? I am receiving sound issues when going through my USB DAC. I am going through a single USB port to a hub. I don't know if I am doing it correctly. Also, is there anyway to tell if I appended the string to intrid correctly?

 

You need to pass through a USB controller for sound to work  correctly when using USB sound cards/DAC. All the sound cards/DAC have had the same problem. You might want to check out archdrafts guide on how to pass through a whole controller here

 

Hey Saarg!

 

I do currently have my USB passthrough functioning with this. it passes through my whole controller. for some reason the qemu from archedraft wasn't functioning for me. It would always revert back to this style. I just knew the address of the PCI for my usb and copy pasted the hostdev from my GPU assignment, and gave it its own slot

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>

 

I'm just not sure if the CPUs are pinned correctly. OP said that we shouldn't change because it would change by itself

<cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>

 

but I changed mine to

 
<cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>

because I am using 3 CPUs and 6 threads (I think thats right)

 

I messed around with the buffer of my DAC a little bit but still getting some underruns and some clipping.

 

still tweaking about...

 

Let's see if we can fix this. Let us take it from the start.

 

1. Find your core pairs (core and hyperthread) in the dashboard. Only if you have a CPU with hyperthreading. You will see that in the CPU Speed field you should see core 0 / 16 (In my case). The numbers make your pairs. So in this case, the pair is core 0 and 16

2. Knowing the pairs you can start with the emulatorpin cores. I like to leave core 0 and its pair to unraid, so I would use the next available pair. Remember to add the pair to isolcpus= parameter in syslinux.cfg. Should look like below.

 

  append isolcpus=1,17 initrd=/bzroot

 

3. Find the pairs for the cores you want to let your VM use. Also add these to isolcpus like above.

 

4. Now we need to specify the correct amount of cores in the xml template. Start with the below first. You should input six as that is the number of cores you want. I use core for both the real core and the hyperthread, as the VM only see a virtual CPU (vcpu).

 

<vcpu placement='static'>6</vcpu>

 

Then you change the last part (cpuset) to the cores you found in step 3.

 

    <vcpupin vcpu='0' cpuset='12'/>
    <vcpupin vcpu='1' cpuset='13'/>
    <vcpupin vcpu='2' cpuset='14'/>
    <vcpupin vcpu='3' cpuset='15'/>
    <vcpupin vcpu='4' cpuset='28'/>
    <vcpupin vcpu='5' cpuset='29'/>

 

Now add the emulatorpin cores you chose in step 2.

 

<emulatorpin cpuset='1,17'/>

 

The last thing is to change the CPU topology. You should set sockets and threads to 1 and change cores to the number of cores you gave the VM, which is 6.

 

  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='6' threads='1'/>
  </cpu>

 

That was it for the CPU. Now let us try the USB controller.

Just to be sure you have it correctly, remove the old passthrough and add the below.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
    </hostdev>

 

You might need to hide the usb controller from unraid, so it doesn't use any of the devices before the VM is started. In most cases this is no problem, but I have seen some cases where it messed up the pass through.

It looks like this is one of the onboard controllers and then it might share the same vendor/product id as the other controllers you have.  But for this I need the PCI device list in Tools --> System Devices if you are on unRAID 6.2 or the output of the below command (SSH to the server) if you are on 6.1.9.

 

lspci -nn

 

After you have supplied the info, we can continue with hiding the controller.

Share this post


Link to post

Hey,

 

I just registered since the Forum here seems to be most active when discussing kvm and cpu pinning.

My setup consists of a e5-2670 with 32G ram - which I will be using for a kvm host with several VMs and LXC containers. Additionally I plan on creating a VM utilizing GPU (PCI) passthrough for gamestreaming.

 

With this setup I`d like to allocate resources more dynamically. By that I mean, that depending if the "game vm" is active or not, I want to allocate the real cpu cores to the vCPUs in the setup.

 

Example:

Game VM OFF: every VM running has their own CPU pinning spread accross cores 1-8 (1-16 with HT)

Game VM ON: game VM has exclusive right to cores 5-8; all other VMs will only be allowed to use cores 1-4

 

Did anyone ever attempt something similar? Do you know if there is any tool for this? (Besides using libvirt / virsh

 

I have two concepts in mind:

1) define two working sets / xml files per VM (one for gamevm on; one for gamevm off) and switch between them depending on the state

 

2)

- have a tool read the current (cpu) configuration of the vms before "game vm" is started

- limit running vms to cores 1-4 and then start game vm

- after game vm stops, restore initial configuration

 

Any thoughts or ideas on this? How do you manage your cpu pinning with running vms?

Share this post


Link to post

I see what you are trying to do.

 

Using multiple sets of xmls is going to be the best senario. I currently do this for various vms. I am lucky enough to have a cpu with a high core count.

Of the14 cores available i isolate 8 of them from the host linux system using

isolcpus=

for example I have a vm with 12 cores assigned, 8 cores assigned and 4 cores assigned using 3 different xmls and one vdisk. I will use the 12 core vm when the workload requires it. 8 for the gaming  but the 4 cores for general stuff.

 

Isolating 8 cores leaves 6 cores for the host and for dockers etc. The 8 cores that are isolated are assigned to a gaming vm so i know they will be purely used by the vm even if emby starts trancoding or a download is unzipped whilst the vm is running.

The 8 cores are probably overkill for gaming to be honest.

When using the 12 core vm i have to take another 4 cores from the host system that are not isolated, so there is a chance those cores will be shared by other processes.On all my pins i emulatorpin the emulation tasks to 2 cores that are not assigned to vms. This works well for my needs.

 

As far as i am aware there is no way to reassign cores to running vms etc. Anyway its not hard to stop and start vms manually and get the same result.

 

Share this post


Link to post

Thanks for your thought. Actually the xeon e5-2670 has quite a number of cores and the power should be plenty for using it both for work (gitlab+ci / staging area / low traffic websites) and leisure (mediaserver/plex + gaming).

 

Using multiple sets of xmls is going to be the best senario. I currently do this for various vms. I am lucky enough to have a cpu with a high core count.

Of the14 cores available i isolate 8 of them from the host linux system using

isolcpus=

for example I have a vm with 12 cores assigned, 8 cores assigned and 4 cores assigned using 3 different xmls and one vdisk. I will use the 12 core vm when the workload requires it. 8 for the gaming  but the 4 cores for general stuff.

 

Thanks for explaining your setup - it is however not what I am looking to achieve. In your setup you change the vCPUs or virtual cores of your VM. I don`t want to change the core count of my VMs - I want to change what physical cores the vCPUs can utilize.

 

Example Setup: assuming VM1 (2 vCPUs), VM2 (4vCPUs), GamingVM (4 vCPUs),  e5-2670 with 8 physical cores (I omit the HT cores for this example)

 

When GamingVM is OFF: VM1 and VM2 are free to use any of the 8 physical cores for their work.

When GamingVM is ON: VM1 and VM2 are only allowed to use phyiscal cores 1-4; GamingVM is allowed to only use physical cores 5-8

 

There is no reason for having vCPU count == pysical core count. You can (and should) over commit resources, for increasing overall utilization

 

As far as i am aware there is no way to reassign cores to running vms etc. Anyway its not hard to stop and start vms manually and get the same result.

 

I`d like to avoid (re-)starting VMs.

Reason basically is, that some services should ideally be without downtime (webservices, gitlab ...). If the "gaming vm" resources are not in use - i`d like to utilize them for CI slaves or other work related tasks.

 

 

As far as I could read, libvirt (and the virsh command line tool) allows for changing numa  values (in this case vcpupin) while the VM is live. see virsh documentation.

There are some limitations to changing live parameters:

for example the vCPU count can be increased, but only to a previously specified maximum - so essentially I could add/remove vCPUs - but I am not looking into changing the amount of virtual Cores/virtual CPUs.

I`d rather scale horizontal (more VMs) than vertical (more vCPUs per VM).

 

So the important part for me is, to adjust cpu pinning/affinity (the pysical cores that vCPUs are allowed to use) on live VMs when certain conditions are met and not change the amount of vCPUs (or virtual cores). Would be great if there is already a tool to automate this process. Maybe someone else has already attempted something similar or already a working setup for this scenario? I guess this use case would also be something others here might be interested?

Share this post


Link to post

When GamingVM is OFF: VM1 and VM2 are free to use any of the 8 physical cores for their work.

When GamingVM is ON: VM1 and VM2 are only allowed to use phyiscal cores 1-4; GamingVM is allowed to only use physical cores 5-8

 

Ok yes i understand now. So assigning 8 cores but being able to scale those back on the start of another vm. Thats a very interesting idea and i didnt know that was possible. It would certainly be very useful. I had a similar idea a while back about isolcpus capability post-boot to be able to isolate and release cpus to the host without reboot. Having spoken to limetech this is something they have been investigating but it isnt in Linux's capability set and as such would be only be possible manually manipulating some things in the user space for this but it isnt something they are currently working on.

 

So yes any info on how to implement what you are talking about here would be most welcome.

 

Share this post


Link to post

So what is the general consensus on how much overhead or performance we lose on the CPU compared to barebone. I have done some low resolution testing and it seems somewhere in the 30+% loss in vm. This is not usually noticed when gaming in higher resolution and detail but I am starting to notice cpu limitation in a few games. I have confirmed all of my settings are correct so I am sure that isn't the problem. Maybe one of the developers could give some insight into this? 

Share this post


Link to post

Hey All!

 

I was wondering if there are any audio engineers out there using this method? I am receiving sound issues when going through my USB DAC. I am going through a single USB port to a hub. I don't know if I am doing it correctly. Also, is there anyway to tell if I appended the string to intrid correctly?

 

You need to pass through a USB controller for sound to work  correctly when using USB sound cards/DAC. All the sound cards/DAC have had the same problem. You might want to check out archdrafts guide on how to pass through a whole controller here

 

Hey Saarg!

 

I do currently have my USB passthrough functioning with this. it passes through my whole controller. for some reason the qemu from archedraft wasn't functioning for me. It would always revert back to this style. I just knew the address of the PCI for my usb and copy pasted the hostdev from my GPU assignment, and gave it its own slot

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>

 

I'm just not sure if the CPUs are pinned correctly. OP said that we shouldn't change because it would change by itself

<cpu mode='host-passthrough'>
    <topology sockets='1' cores='2' threads='2'/>
  </cpu>

 

but I changed mine to

 
<cpu mode='host-passthrough'>
    <topology sockets='1' cores='3' threads='6'/>
  </cpu>

because I am using 3 CPUs and 6 threads (I think thats right)

 

I messed around with the buffer of my DAC a little bit but still getting some underruns and some clipping.

 

still tweaking about...

 

Let's see if we can fix this. Let us take it from the start.

 

1. Find your core pairs (core and hyperthread) in the dashboard. Only if you have a CPU with hyperthreading. You will see that in the CPU Speed field you should see core 0 / 16 (In my case). The numbers make your pairs. So in this case, the pair is core 0 and 16

2. Knowing the pairs you can start with the emulatorpin cores. I like to leave core 0 and its pair to unraid, so I would use the next available pair. Remember to add the pair to isolcpus= parameter in syslinux.cfg. Should look like below.

 

  append isolcpus=1,17 initrd=/bzroot

 

3. Find the pairs for the cores you want to let your VM use. Also add these to isolcpus like above.

 

4. Now we need to specify the correct amount of cores in the xml template. Start with the below first. You should input six as that is the number of cores you want. I use core for both the real core and the hyperthread, as the VM only see a virtual CPU (vcpu).

 

<vcpu placement='static'>6</vcpu>

 

Then you change the last part (cpuset) to the cores you found in step 3.

 

    <vcpupin vcpu='0' cpuset='12'/>
    <vcpupin vcpu='1' cpuset='13'/>
    <vcpupin vcpu='2' cpuset='14'/>
    <vcpupin vcpu='3' cpuset='15'/>
    <vcpupin vcpu='4' cpuset='28'/>
    <vcpupin vcpu='5' cpuset='29'/>

 

Now add the emulatorpin cores you chose in step 2.

 

<emulatorpin cpuset='1,17'/>

 

The last thing is to change the CPU topology. You should set sockets and threads to 1 and change cores to the number of cores you gave the VM, which is 6.

 

  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='6' threads='1'/>
  </cpu>

 

That was it for the CPU. Now let us try the USB controller.

Just to be sure you have it correctly, remove the old passthrough and add the below.

 

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x1d' function='0x0'/>
      </source>
    </hostdev>

 

You might need to hide the usb controller from unraid, so it doesn't use any of the devices before the VM is started. In most cases this is no problem, but I have seen some cases where it messed up the pass through.

It looks like this is one of the onboard controllers and then it might share the same vendor/product id as the other controllers you have.  But for this I need the PCI device list in Tools --> System Devices if you are on unRAID 6.2 or the output of the below command (SSH to the server) if you are on 6.1.9.

 

lspci -nn

 

After you have supplied the info, we can continue with hiding the controller.

 

 

My Pairs Are 0/1 2/3 4/5 ... 26/27

 

  append isolcpus=1,17 initrd=/bzroot <-- am I suppose to isolcpus with all that i want isolated? 8/9...26-27 ?

 

 

vcpu placement is static and 6

 

 

4. goes back to question 2. should I only be isolating the pin (my case 8/9)

 

will change to 6 cores 1 thread, I thought it would be 3 cores with 2 threads initially...

 

USBs are working correctly.

 

Thanks for your help! I just want to verify step 2 before letting you know if it worked!

 

 

Share this post


Link to post

So what is the general consensus on how much overhead or performance we lose on the CPU compared to barebone. I have done some low resolution testing and it seems somewhere in the 30+% loss in vm. This is not usually noticed when gaming in higher resolution and detail but I am starting to notice cpu limitation in a few games. I have confirmed all of my settings are correct so I am sure that isn't the problem. Maybe one of the developers could give some insight into this?

 

I get about 3-5% loss in cpu performance, but I'm also using older business class hardware.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.