Xeon 5660 Stability issues


Recommended Posts

Hey,

 

Hoping someone can point me in the right direction or offer some suggestions, I have a rig with duel Xeon 5660 and I am having some realy bad cpu lag, as seen below:

 

image.png.2a092f7c5c4a7cfe53a99ee4a85340fe.png

 

I have tried disablin multithreading and manually assigning the cores and setting emulatorpin my current cpu tune is:

 

  <vcpu placement='static'>4</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='10'/>
    <vcpupin vcpu='3' cpuset='11'/>
    <emulatorpin cpuset='6-7'/>
  </cputune>

 

I am testing without any passthroughs as well. My cpu assignments are as follows (Note I am hyperthreading is off at the moment):

 

PU Thread Pairings

Single:	cpu 0
Single:	cpu 1
Single:	cpu 2
Single:	cpu 3
Single:	cpu 4
Single:	cpu 5
Single:	cpu 6
Single:	cpu 7
Single:	cpu 8
Single:	cpu 9
Single:	cpu 10
Single:	cpu 11

 

Any help appreicated :) 

Link to comment

Did you play around with the MSI mode utility? For me to get video playback to the point it was acceptable on my vm I had to set my sound card and video card to use MSI interrupts and then reboot. Much better and way more stable.  See below. It's not perfect, but much better. I still use the SMT threads (6 cores 12 threads) but I'm also on an AMD 3900x. 

 

There are threads on these forums disusing this and other solutions to try.

 

image.thumb.png.6a9146349b049769f156d17113acdefe.png

MSI_util_v2.zip

Edited by Chess
spelling
Link to comment
20 hours ago, Chess said:

Did you play around with the MSI mode utility? For me to get video playback to the point it was acceptable on my vm I had to set my sound card and video card to use MSI interrupts and then reboot. Much better and way more stable.  See below. It's not perfect, but much better. I still use the SMT threads (6 cores 12 threads) but I'm also on an AMD 3900x. 

 

There are threads on these forums disusing this and other solutions to try.

 

image.thumb.png.6a9146349b049769f156d17113acdefe.png

MSI_util_v2.zip 12.96 kB · 0 downloads

 

Hey thanks, I have looked through a lot of other threads and I have managed since posting this, to get some improvement. by setting host mode instead of pass through. It made a massive difference but my latency was still a problem. I have since re-added my passthrough for GPU and USB keyboard.

 

My current config is:

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='4'>
  <name>VM Desktop</name>
  <uuid>c8646900-bc2c-923c-4a81-e43dbecec448</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>33554432</memory>
  <currentMemory unit='KiB'>33554432</currentMemory>
  <vcpu placement='static'>6</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='7'/>
    <vcpupin vcpu='2' cpuset='8'/>
    <vcpupin vcpu='3' cpuset='9'/>
    <vcpupin vcpu='4' cpuset='10'/>
    <vcpupin vcpu='5' cpuset='11'/>
    <emulatorpin cpuset='0-4'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-5.0'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/c8646900-bc2c-923c-4a81-e43dbecec448_VARS-pure-efi.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' dies='1' cores='6' threads='1'/>
    <cache mode='passthrough'/>
    <feature policy='disable' name='vmx'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source file='/mnt/fast_vms/SSD/MK-VMDESKTOP/vdisk1.img' index='2'/>
      <backingStore/>
      <target dev='hdc' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/user/isos/virtio-win-0.1.173-2.iso' index='1'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:b4:fb:1a'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/0'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-4-VM Desktop/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-gb'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0f' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0f' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x413c'/>
        <product id='0x2501'/>
        <address bus='8' device='2'/>
      </source>
      <alias name='hostdev2'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

 

I used the MSI untility you send (Thanks for that, I used one from github but it didnt have a gui) and my results now look like this:

 

image.thumb.png.9a1c0cd865c7588f45d17c8417fc5d15.png

 

I'd say it improved things by about 15-20% on my last test. So the utility worked. 

 

Do you have any other ideas of how I can get it all under 1000?

Edited by McKnight
Link to comment
25 minutes ago, McKnight said:

 

Hey thanks, I have looked through a lot of other threads and I have managed since posting this, to get some improvement. by setting host mode instead of pass through. It made a massive difference but my latency was still a problem. I have since re-added my passthrough for GPU and USB keyboard.

 

My current config is:

 

I used the MSI untility you send (Thanks for that, I used one from github but it didnt have a gui) and my results now look like this:

 

I'd say it improved things by about 15-20% on my last test. So the utility worked. 

 

Do you have any other ideas of how I can get it all under 1000?

 

This is not what you want to hear, but alas no. I still get spikes, but I no longer notice them and games don't studder on me. Let me see if I can find the thread(s) that I used to get here. I also noticed that being on the 6.9 betas made it better, but if you read the second thread you will see that unRaid is not 100% leaving the isolated cores alone, even when the VM is powered down. I'm on the VM right now and in task manager it says that my CPU usage is around 10%, but unRaid is showing all of the VMs used cores around 20 to 25% usage. The 3900x has enough power with some of the fixes in the below threads to "power" through the extra load, but older CPUs might not be able to. It's also worse on Ryzen 2 CPUs at the moment due to a bug in version of quem that the current beta is using and I can't do host pass-through at the moment. 

 

I suspect your CPU has Hyper Threading, correct? Hyper Threading can make the issue worse, so while I don't want to say turn off HT, but maybe run a test with it off and see if that improves the situation. The VM is not aware of HT and treats all cores as full CPUs and does not schedule threads properly. I believe you can tell the VM the actual CPU topology (X Cores X Threads) but I never dug too deep into it. I suspect if I did it :right" I'd see some improvement in DPI latency, but I'm at a point that works for me.

 

Below are a few of the threads that helped me get to a playable state. Before doing all this games would studder very badly and was unplayable and for me this system was indented for game streaming, PLEX, backing up physical systems and a number of other minor functions.

 

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.