Performance Improvements in VMs by adjusting CPU pinning and assignment


Recommended Posts

Linux tends to favor the low numbered cpus, so I would expect the first pair would be loaded more.  My recommendation is based on an idea I have presented to LT as a feature request to emulator pin the first pair in all VMs using the VM manager.  The idea is that in a lot of cases doing this would remove latency from some VMs and cut down on the the gyrations needed to get VMs running better.  I was looking for feedback like yours to see if my recommendation has merit.

 

Yes, you can emulatorpin other cpus.  I have been unable to find anything on the internet about how much load the emulator tasks on a VM load cpus, but my gut feel is not much.

 

Linux will use additional cpus as it needs them, so any that are not isolated will be available for Linux.

 

Don't get too concerned about cpu frequency.  The cpu scaling driver/governor that adjusts the cpu frequency can switch up the cpu frequency under a relatively small load.  The Intel pstate driver I am using on my Xeon will ramp up to full speed under a 10-15% load.  Full speed on a cpu does not necessarily mean it is fully loaded.

 

Good to know, and good feedback as well!

 

When I got home from work I started doing some testing on my home 12 core xeon. I tried out some different setups with my 2 Win10 gaming VMs and I have to say that I agree with your suggestion of using emulatorpin for the first pair only on the VMs. Its a bit late here now, so I hope what I write will make sense, but here goes.

 

Keeping in mind I have hyper-v on in all these instances as I know some people have had issues using it.

After adding isolcpus for all but the 1st two pairs, and using my original vcpupin setups, I tried emulatorpin on the first pair solely, and then again with the first two pairs combined. I then tested the visual performance and checked fps in game for Warthunder and also Black Desert. I tried different vcpupin setups, using 5 pairs, 4 pairs, 2 pairs, and repeated tests of the above without allocating the 2nd thread. In every test using emulatorpin on only the first pair, it gave better performance and stability for me as compared with using the first two pairs. Running the VMs without the 2nd thread allocated with vcpupin, gave me a pretty bad stutter.

 

I didn't seem to have any over allocation issues and ultimately found that 5cores 2 threads with the 1st pair emulatorpin'd is the sweet spot for my VMs at the moment. 4 pairs caused stuttering in game. 2 pairs had roughly the same fps as 5pairs, but would have latency if the cpu spiked; war thunder wouldn't even load.

Link to comment

Linux tends to favor the low numbered cpus, so I would expect the first pair would be loaded more.  My recommendation is based on an idea I have presented to LT as a feature request to emulator pin the first pair in all VMs using the VM manager.  The idea is that in a lot of cases doing this would remove latency from some VMs and cut down on the the gyrations needed to get VMs running better.  I was looking for feedback like yours to see if my recommendation has merit.

 

Yes, you can emulatorpin other cpus.  I have been unable to find anything on the internet about how much load the emulator tasks on a VM load cpus, but my gut feel is not much.

 

Linux will use additional cpus as it needs them, so any that are not isolated will be available for Linux.

 

Don't get too concerned about cpu frequency.  The cpu scaling driver/governor that adjusts the cpu frequency can switch up the cpu frequency under a relatively small load.  The Intel pstate driver I am using on my Xeon will ramp up to full speed under a 10-15% load.  Full speed on a cpu does not necessarily mean it is fully loaded.

 

Good to know, and good feedback as well!

 

When I got home from work I started doing some testing on my home 12 core xeon. I tried out some different setups with my 2 Win10 gaming VMs and I have to say that I agree with your suggestion of using emulatorpin for the first pair only on the VMs. Its a bit late here now, so I hope what I write will make sense, but here goes.

 

Keeping in mind I have hyper-v on in all these instances as I know some people have had issues using it.

After adding isolcpus for all but the 1st two pairs, and using my original vcpupin setups, I tried emulatorpin on the first pair solely, and then again with the first two pairs combined. I then tested the visual performance and checked fps in game for Warthunder and also Black Desert. I tried different vcpupin setups, using 5 pairs, 4 pairs, 2 pairs, and repeated tests of the above without allocating the 2nd thread. In every test using emulatorpin on only the first pair, it gave better performance and stability for me as compared with using the first two pairs. Running the VMs without the 2nd thread allocated with vcpupin, gave me a pretty bad stutter.

 

I didn't seem to have any over allocation issues and ultimately found that 5cores 2 threads with the 1st pair emulatorpin'd is the sweet spot for my VMs at the moment. 4 pairs caused stuttering in game. 2 pairs had roughly the same fps as 5pairs, but would have latency if the cpu spiked; war thunder wouldn't even load.

 

Good information that confirms what I felt was the best approach.  I have suggested that LT change the VM manager to emulatorpin every VM to the first cpu pair and not allow assigning the first pair to any VM.

Link to comment

dlandon,

 

I need some help figuring out how I should assign my cores. Been trying to figure out how to configure, but it is late, and I can't seem to grasp it.

 

Here is a screenshot of my cpu parings:

yU2PHUO.png

Using x2 Xeon 2670 v1 CPUS 8 cores each.

64GB RAM

1 Nvidia GTX 970

unRAID 6.19

 

I have 2 VMs at the moment,

VM1 - Windows 10 - handles a filezilla server, my uTorrents and an Emby server. - Windows 10

VM2 - Windows 10 - is intended to be my gaming rig but I am getting horrible performance with it and my GTX 970. Really really bad FPS.

 

The focus is getting the gaming rig working properly as I have seen videos of people doing it this way with only losing about 1-2% performance.

 

Is this a problem with 6.19 as I see alot of you guys using the newer beta?

 

How should I configure this?

 

Thank you for your help.

 

//Uroth

 

Link to comment

dlandon,

 

I need some help figuring out how I should assign my cores. Been trying to figure out how to configure, but it is late, and I can't seem to grasp it.

 

Here is a screenshot of my cpu parings:

yU2PHUO.png

Using x2 Xeon 2670 v1 CPUS 8 cores each.

64GB RAM

1 Nvidia GTX 970

unRAID 6.19

 

I have 2 VMs at the moment,

VM1 - Windows 10 - handles a filezilla server, my uTorrents and an Emby server. - Windows 10

VM2 - Windows 10 - is intended to be my gaming rig but I am getting horrible performance with it and my GTX 970. Really really bad FPS.

 

The focus is getting the gaming rig working properly as I have seen videos of people doing it this way with only losing about 1-2% performance.

 

Is this a problem with 6.19 as I see alot of you guys using the newer beta?

 

How should I configure this?

 

Thank you for your help.

 

//Uroth

 

This strategy works equally well with 6.1.9 as 6.2.

 

Some recommendations:

- I assume your VMs are on an SSD disk.

- You should give the VMs no more that 8GB of memory for the moment.

- Don't use balloon memory.

 

I would do it this way:

isolcpu 8-15,24-31

 

The gaming VM:

pin cpus 12,13,14,15,28,29,30,31

emulator pin 0,16

 

The second VM:

pin cpus 8,9,10,11,24,25,26,27

emulator pin 0,16

 

I'm not a fan of over provisioning cpus as it can counterproductive.  In your case you have an enormous number of cpus available, so try it this way first.

 

Also try the setting the 'Hyper-V' setting on and off in the VM manager.  On is the best setting, but might work better if off.

 

Let me know how it works out with this configuration and some specifics if it isn't working as you expect.  I have some more ideas if you continue to see issues.

Link to comment

dlandon, still the same issue. The VMs feels sluggish, and the gaming VM is getting between 1fps-20fps on games where I should be getting close to 80-100fps.

 

I even have the two VMs on different SSDs.

 

 

Here is my syslinux.cfg: (is this correct?)

label unRAID OS

  menu default

 

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot

  append isolcpus=8,9,10,11,12,13,14,15,24,25,26,27,28,29,30,31 initrd=/bzroot

 

 

and here is the xml settings for the Gaming-VM

<cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='0,16'/>

 

How do I turn off balloon memory?

 

When I try to use Hyper-V the computer bluescreen on startup.

 

Tried running only the Gaming-VM, same problem.

 

 

*EDIT* Should I insert another SSD, not boot into unRAID, install Windows 10 on the SSD and see what happens when I try and run the same games in a non virtual windows install?

Link to comment

dlandon, still the same issue. The VMs feels sluggish, and the gaming VM is getting between 1fps-20fps on games where I should be getting close to 80-100fps.

 

I even have the two VMs on different SSDs.

 

 

Here is my syslinux.cfg: (is this correct?)

label unRAID OS

  menu default

 

  kernel /bzimage

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream initrd=/bzroot

  append isolcpus=8,9,10,11,12,13,14,15,24,25,26,27,28,29,30,31 initrd=/bzroot

 

 

and here is the xml settings for the Gaming-VM

<cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='0,16'/>

 

How do I turn off balloon memory?

 

When I try to use Hyper-V the computer bluescreen on startup.

 

Tried running only the Gaming-VM, same problem.

 

 

*EDIT* Should I insert another SSD, not boot into unRAID, install Windows 10 on the SSD and see what happens when I try and run the same games in a non virtual windows install?

 

Several problems:

Fix this:

  append intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream isolcpus=8-15,24-31 initrd=/bzroot

 

I don't think the cpus got isolated properly.  Why are you adding all the other Linux options?  Generally those are not necessary.

 

The SSDs are probably not a problem.  Don't be overly concerned with that right now.

 

Is the VM booting in OVMF?

 

Start with getting the VM working this way:

Add the isolcpus properly to the syslinux.cfg.

Booting with OVMF.  The GPU has to support UEFI boot.

Don't add any of the Linux options in the syslinux.cfg and just get the VM working with the GPU.

Set min and max memory the same.  That will turn off ballooning.

Link to comment

Ok, sorry for the delay, here is an update:

I still have the same problems of the games running poorly and the VM feeling slugish.

 

.cfg:

label unRAID OS

  menu default

  kernel /bzimage

  append isolcpus=8-15,24-31 initrd=/bzroot

 

xml:

<cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='0,16'/>

 

 

I have noticed after I have started the VM and later when I turn it off,  <emulatorpin cpuset='0,16'/> is removed from the xml.

Using OVMF as the bios.

 

 

Tested installing Windows 10 directly to the SSD and booted into windows. Works fine. Why isn't the VM running well :/ Really want this to work.

 

And now I need to reinstall the VM again, after a crash I am stuck in a OVMF boot loop.

 

 

Thank you again dlandon for trying to help me.

Link to comment

The SSD is mounted outside the array, using the Unassigned Devices plugin. Mounted as /mnt/disks/Corsair_Force_GS_240G. Formated as XFS.

 

XML

<domain type='kvm'>

  <name>Windows 10 - Gaming</name>

  <uuid>009382e0-d3d1-2e08-f785-8c0dab461393</uuid>

  <metadata>

    <vmtemplate name="Custom" icon="windows.png" os="windows"/>

  </metadata>

  <memory unit='KiB'>8388608</memory>

  <currentMemory unit='KiB'>8388608</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

    <loader type='pflash'>/usr/share/qemu/ovmf-x64/OVMF-pure-efi.fd</loader>

  </os>

  <features>

    <acpi/>

    <apic/>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='8' threads='1'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='rtc' tickpolicy='catchup'/>

    <timer name='pit' tickpolicy='delay'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/disks/Corsair_Force_GS_240GB/Windows 10 - Gaming/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/PROGRAMS/IMAGES/Windows_10_x64.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/PROGRAMS/IMAGES/virtio-win-0.1.117.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:a0:11:6c'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10 - Gaming.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc31c'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x0037'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

Link to comment
  • 2 weeks later...

I just added some additional information to the OP.  I see some trying to get performance improvements from a VM when they have not configured the VM properly to start with.  In general, you must boot a Windows VM in OVMF, and not over provision CPUs and memory.  Get a good performing VM set up before trying to pin CPUs for additional performance.

Link to comment

The SSD is mounted outside the array, using the Unassigned Devices plugin. Mounted as /mnt/disks/Corsair_Force_GS_240G. Formated as XFS.

 

XML

<domain type='kvm'>

  <name>Windows 10 - Gaming</name>

  <uuid>009382e0-d3d1-2e08-f785-8c0dab461393</uuid>

  <metadata>

    <vmtemplate name="Custom" icon="windows.png" os="windows"/>

  </metadata>

  <memory unit='KiB'>8388608</memory>

  <currentMemory unit='KiB'>8388608</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>8</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

  </cputune>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

    <loader type='pflash'>/usr/share/qemu/ovmf-x64/OVMF-pure-efi.fd</loader>

  </os>

  <features>

    <acpi/>

    <apic/>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='8' threads='1'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='rtc' tickpolicy='catchup'/>

    <timer name='pit' tickpolicy='delay'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/disks/Corsair_Force_GS_240GB/Windows 10 - Gaming/vdisk1.img'/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/PROGRAMS/IMAGES/Windows_10_x64.iso'/>

      <target dev='hda' bus='ide'/>

      <readonly/>

      <boot order='2'/>

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>

    </disk>

    <disk type='file' device='cdrom'>

      <driver name='qemu' type='raw'/>

      <source file='/mnt/user/PROGRAMS/IMAGES/virtio-win-0.1.117.iso'/>

      <target dev='hdb' bus='ide'/>

      <readonly/>

      <address type='drive' controller='0' bus='0' target='0' unit='1'/>

    </disk>

    <controller type='usb' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'/>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:a0:11:6c'/>

      <source bridge='br0'/>

      <model type='virtio'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10 - Gaming.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <hostdev mode='subsystem' type='pci' managed='yes'>

      <driver name='vfio'/>

      <source>

        <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>

      </source>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x046d'/>

        <product id='0xc31c'/>

      </source>

    </hostdev>

    <hostdev mode='subsystem' type='usb' managed='yes'>

      <source>

        <vendor id='0x1532'/>

        <product id='0x0037'/>

      </source>

    </hostdev>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

if you are using the ssd exclusively for the vm, i should passthrough the disk then install the os directly to the disk rather than use a vdisk

Link to comment

Ok, I have now installed the VM directly on the SSD by passing through the SSD, so I am not using a Vdisk.

I am also running the latest beta version of unRAID.

 

Tried out 3DMARK on the VM and I am getting very low scores on the CPU part when I use:  http://www.3dmark.com/fs/8904521

Getting around 6650 cpu score.

 

<vcpupin vcpu='0' cpuset='12'/>

    <vcpupin vcpu='1' cpuset='13'/>

    <vcpupin vcpu='2' cpuset='14'/>

    <vcpupin vcpu='3' cpuset='15'/>

    <vcpupin vcpu='4' cpuset='28'/>

    <vcpupin vcpu='5' cpuset='29'/>

    <vcpupin vcpu='6' cpuset='30'/>

    <vcpupin vcpu='7' cpuset='31'/>

    <emulatorpin cpuset='0,16'/>

 

Tried it both with <emulatorpin cpuset='0,16'/> and without it. 

 

When I try with:

<vcpupin vcpu='0' cpuset='4'/>

    <vcpupin vcpu='1' cpuset='5'/>

    <vcpupin vcpu='2' cpuset='6'/>

    <vcpupin vcpu='3' cpuset='7'/>

    <vcpupin vcpu='4' cpuset='16'/>

    <vcpupin vcpu='5' cpuset='17'/>

    <vcpupin vcpu='6' cpuset='18'/>

    <vcpupin vcpu='7' cpuset='19'/>

 

I get a cpu score of around 8850, better, but I know it should be able to score higher.

http://www.3dmark.com/fs/8904680

 

I am still using:

label unRAID OS

  menu default

  kernel /bzimage

  append isolcpus=8-15,24-31 initrd=/bzroot

 

as I was instructed to use.

 

The gfx is getting good scores, and I can actually run games now, but I want to get the best performance out of this.

 

Thanks for the help so far.

Link to comment

Using x2 Xeon 2670 v1, 8 cores each.

64GB RAM

1 Nvidia GTX 970

unRAID 6.2.0-beta22

 

I guess it is time to fix a Build signature.

 

 

Scroll back 1 page in this thread, I have specified my setup there.

 

What are your thread pairings? As you have 2 cpus i would isolate all cores/threads from cpu 2 the assign that whole physical cpu to the vm then pin the emulator pin to a core on the first cpu.

Link to comment

Here are my pairings:

0 <===> 16

1 <===> 17

2 <===> 18

3 <===> 19

4 <===> 20

5 <===> 21

6 <===> 22

7 <===> 23

8 <===> 24

9 <===> 25

10 <===> 26

11 <===> 27

12 <===> 28

13 <===> 29

14 <===> 30

15 <===> 31

 

And current:

label unRAID OS

  menu default

  kernel /bzimage

  append isolcpus=8-15,24-31 initrd=/bzroot

 

Never really understood the paring part.

Link to comment

Here are my pairings:

0 <===> 16

1 <===> 17

2 <===> 18

3 <===> 19

4 <===> 20

5 <===> 21

6 <===> 22

7 <===> 23

8 <===> 24

9 <===> 25

10 <===> 26

11 <===> 27

12 <===> 28

13 <===> 29

14 <===> 30

15 <===> 31

 

Never really understood the paring part.

 

the pairing refers to the threads on each core.

So for example 0 and 16 are core 1.

7 and 23 are core 8. These would be on cpu 1

8 and 24 would be core 1 on cpu 2 (or core 9)

15 and 31 core 8 on cpu 2 (or core 16)

 

It is best to use thread pairs from each core or else that core can have a thread used from another process.

 

I would do the following

 

append isolcpus=8-15, 24-31 initrd=/bzroot

 

This will isolate the whole of your second cpu. So these are the cores you want to assign.

 

so try this

<vcpu placement='static'>16</vcpu>
  <cputune>
<vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='10'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <vcpupin vcpu='4' cpuset='12'/>
    <vcpupin vcpu='5' cpuset='13'/>
    <vcpupin vcpu='6' cpuset='14'/>
    <vcpupin vcpu='7' cpuset='15'/>
    <vcpupin vcpu='8' cpuset='24'/>
    <vcpupin vcpu='9' cpuset='25'/>
    <vcpupin vcpu='10' cpuset='26'/>
    <vcpupin vcpu='11' cpuset='27'/>
    <vcpupin vcpu='12' cpuset='28'/>
    <vcpupin vcpu='13' cpuset='29'/>
    <vcpupin vcpu='14' cpuset='30'/>
    <vcpupin vcpu='15' cpuset='31'/>
    <emulatorpin cpuset='0-1,16-17'/>
</cputune>

 

This will assign the whole of your second cpu to the vm which is isolated from the host os.

The emulator pin is on core 1 and 2 on cpu 1

 

I know the general advise is to use only core 1 for emulator pin, but that is normally people who only have a total of 4 cores in their server, but i think if you have alot of cores, then 2 is better.

I have 14 cores and do a similar thing with my gaming vm by using 8 cores pinned and 2 used for emulator pin and i get very good performance.

 

Hope this gets you where you want to be.

 

 

Link to comment

"VM creation error

 

Maximum CPUs greater than topology limit"

 

When I try your latest edit.

 

um thats strange. That means that the line <vcpu placement='static'>16</vcpu>  the 16 is more than the vcpus listed?

I checked it and cant see a mistake, but guess there must be. So i will write it out again

 

<vcpu placement='static'>16</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='10'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <vcpupin vcpu='4' cpuset='12'/>
    <vcpupin vcpu='5' cpuset='13'/>
    <vcpupin vcpu='6' cpuset='14'/>
    <vcpupin vcpu='7' cpuset='15'/>
    <vcpupin vcpu='8' cpuset='24'/>
    <vcpupin vcpu='9' cpuset='25'/>
    <vcpupin vcpu='10' cpuset='26'/>
    <vcpupin vcpu='11' cpuset='27'/>
    <vcpupin vcpu='12' cpuset='28'/>
    <vcpupin vcpu='13' cpuset='29'/>
    <vcpupin vcpu='14' cpuset='30'/>
    <vcpupin vcpu='15' cpuset='31'/>
    <emulatorpin cpuset='0-1,16-17'/>
  </cputune>

 

if you still get the error with this edit then please edit in gui and assign logical cpus numbers 8 through to 15 and 24 through to 31.

 

press update. then goto edit xml and add

<emulatorpin cpuset='0-1,16-17'/>

the line before </cputune>

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.