FPS drops, stuttering, and other things that make me sad


Recommended Posts

You left the Hyper-V option to Yes, correct?

 

The ASMedia OPROM takes very long to find the SSD, it is fine on bare metal. Then, I can't install windows by passing through that ASMedia controller. It takes about 15-20 minutes to get past the first spinning dots, about 15 minutes to get to selecting a location to install, then it fails by saying it can't be formatted. Also, the USB controller just doesn't work at all.

 

Everything works fine on bare metal. And i've run tests for the SSD and GPU, all fine.

 

Without any controller passthrough, the VM would install just fine on an img on cache.

 

PCIE ACS enabled, X10SL7-F, tried OVMF and its the same, tried Q35, same, tried hyper-v on/off, same... 6.2 Beta 20.

 

I've just unstubbed the devices and i realised that unRAID is not seeing the SATA or USB devices attached to the controller...

 

I am using a PCI-E Riser cable to go from my 8x into a 16x, then put this card in there. I am guessing its that pericom device which appears 3 times in my system devices. I have tried stubbing it to pass through, but it does not appear in the list of devices to pass through. Any ideas here... wonder if its because it repeats 3 times?

 

Diagnostic attached...

 

It would be a shame if it is the card, as its the only USB/SATA combo card i could find!

 

Is there any diagnostic log i could look at to see why its just taking so long to do anything?

tower-diagnostics-20160327-0743.zip

Link to comment
  • Replies 119
  • Created
  • Last Reply

Top Posters In This Topic

Have you tried the MSI fix?

 

http://lime-technology.com/wiki/index.php/UnRAID_6/VM_Guest_Support#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support

 

This was causing lots of stuttering and performance issues for me due to mismanaged interrupts. With this fix I was able to get back to using HPET (no hyperv) and everything is 110% perfect.

 

EDIT: this issue can occur on almost any device. There's some failure in detecting it. When i updated drivers sometimes it detects it, sometimes it doesnt. But always keep a look out on MSI in your host 'lspci'

Link to comment
  • 2 months later...

Spoke too soon. While the latency is massively improved I still get random spikes that ruin any chances of using my TV tuner :(

 

This would indicate that you are probably getting CPU latency because the CPU scaling change affected your TV Tuner.

 

Have you looked at this post to see if any of this might apply to your situation?

 

http://lime-technology.com/forum/index.php?topic=49051.0

Link to comment

I think I've read that post before sadly.

 

If I use something like dpclat, it shows almost continual green with very very low latency but occasionally it'll just spike red. I put my TV tuner in and passed it through to the VM (had to use PCI-E override to do this though). While it works fine, the dpclat app then shows it alternating between green and yellow almost continually the whole time I have a TV channel tuned (only seems to affect HD broadcasts for some reason, I'm assuming due to the larger data rate?).

 

lat.JPG.115e1b58476c4257f6b3060d4d5c8229.JPG

Link to comment

It seems also worth noting that using a Windows 10 VM doesn't seem to have any issues with the same config XML as a Windows 7 desktop.

 

This is my config:

 

<domain type='kvm' id='5'>

  <name>Windows 10</name>

  <uuid>1b1097f5-85a9-aad8-8d51-2b05c558ad0e</uuid>

  <metadata>

    <vmtemplate name="Custom" icon="windows.png" os="windows"/>

  </metadata>

  <memory unit='KiB'>8388608</memory>

  <currentMemory unit='KiB'>8388608</currentMemory>

  <memoryBacking>

    <nosharepages/>

    <locked/>

  </memoryBacking>

  <vcpu placement='static'>2</vcpu>

  <cputune>

    <vcpupin vcpu='0' cpuset='2'/>

    <vcpupin vcpu='1' cpuset='3'/>

    <emulatorpin cpuset='0-1'/>

  </cputune>

  <resource>

    <partition>/machine</partition>

  </resource>

  <os>

    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

  </os>

  <features>

    <acpi/>

    <apic/>

    <hyperv>

      <relaxed state='on'/>

      <vapic state='on'/>

      <spinlocks state='on' retries='8191'/>

    </hyperv>

  </features>

  <cpu mode='host-passthrough'>

    <topology sockets='1' cores='2' threads='1'/>

  </cpu>

  <clock offset='localtime'>

    <timer name='hypervclock' present='yes'/>

    <timer name='hpet' present='no'/>

  </clock>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-system-x86_64</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='writeback'/>

      <source file='/mnt/user/Hybrid/vms/Windows 10/Windows 10/vdisk1.img'/>

      <backingStore/>

      <target dev='hdc' bus='virtio'/>

      <boot order='1'/>

      <alias name='virtio-disk2'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

    </disk>

    <controller type='usb' index='0'>

      <alias name='usb'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>

    </controller>

    <controller type='pci' index='0' model='pci-root'>

      <alias name='pci.0'/>

    </controller>

    <controller type='virtio-serial' index='0'>

      <alias name='virtio-serial0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:af:fc:af'/>

      <source bridge='br0'/>

      <target dev='vnet0'/>

      <model type='virtio'/>

      <alias name='net0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </interface>

    <serial type='pty'>

      <source path='/dev/pts/0'/>

      <target port='0'/>

      <alias name='serial0'/>

    </serial>

    <console type='pty' tty='/dev/pts/0'>

      <source path='/dev/pts/0'/>

      <target type='serial' port='0'/>

      <alias name='serial0'/>

    </console>

    <channel type='unix'>

      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/Windows 10.org.qemu.guest_agent.0'/>

      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>

      <alias name='channel0'/>

      <address type='virtio-serial' controller='0' bus='0' port='1'/>

    </channel>

    <input type='tablet' bus='usb'>

      <alias name='input0'/>

    </input>

    <input type='mouse' bus='ps2'/>

    <input type='keyboard' bus='ps2'/>

    <graphics type='vnc' port='5900' autoport='yes' websocket='5700' listen='0.0.0.0' keymap='en-us'>

      <listen type='address' address='0.0.0.0'/>

    </graphics>

    <video>

      <model type='vmvga' vram='16384' heads='1'/>

      <alias name='video0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </video>

    <memballoon model='virtio'>

      <alias name='balloon0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

    </memballoon>

  </devices>

</domain>

 

My CPU is only a core i5 4460 which as far as I can tell doesn't support hyperthreading. I have isolated core 2 and 2 in my flash config.

Link to comment

Even though your processor is not hyperthreading, the same principles of core pinning can apply.

 

Tell me more about your configuration.

 

What version of unRAID?

 

syslinux.cfg?

 

Windows 7 or Windows 10 VM?

 

What version of virtio drivers are you using?

 

GPU passthrough?

 

Network bonding?  If so what mode.

 

Install this plugin and see if you can't make some adjustments there to clear it up:

 

http://lime-technology.com/forum/index.php?topic=49233.0

Link to comment

Unraid is version 6.1.9

 

Syslinux.cfg:

default /syslinux/menu.c32

menu title Lime Technology

prompt 0

timeout 50

label unRAID OS

  menu default

  kernel /bzimage

  append pcie_acs_override=downstream isolcpus=2,3 initrd=/bzroot

label unRAID OS Safe Mode (no plugins)

  kernel /bzimage

  append initrd=/bzroot unraidsafemode

label Memtest86+

  kernel /memtest

 

Windows 7 and Windows 10 VMs

 

virtio drivers are 0.1.102

 

No GPU passthrough.

 

Network bonding is off.

Link to comment

I've personally had issues with Virtio drivers past 0.1.100, especially on Windows 7.  You might try them and install the Tips and Tweaks plugin and turn off network flow control and disable nic offload.  Set disk caching as recommended in the help for VMs.  This may help if you have a lot of memory.

 

Also take a look at some of the recommendations on the Tips and Tweaks plugin for some other ideas.

 

Issues with latency can come from many different places.

Link to comment
  • 1 month later...

Hi all,

 

I thought I'd share some notes from my experiments. First, my system at the time of these notes:

 

AsRock Z77

Intel® Xeon® CPU E3-1245 V2 @ 3.40GHz (8 cores with hyperthreading)

32 GB of RAM

Unraid 6.2RC2

1 Windows 10 VM, 1 Windows 7 VM, 5 or so Docker containers.

GTX 960 being passed through to the Win10 VM

Supermicro MV8 controller

 

First, the initial setup:

 

- My motherboard put the disk controller and GPU on the same IOMMU group. To fix it, I enabled "Enable PCIe ACS Override"

  in Settings -> VM Manager

- Initially I used the VM config to pass the keyboard and mouse to the VM. But the mouse was periodically pausing for 1-2 seconds. Very annoying. USB passthrough solved it.

- Graphics seemed choppy, like a low frame rate. Enabling MSI interrupts seemed to help.

 

With this setup, I had audio "crackling" listening to youtube. I used the dpc latency tool, and it was all red. It seemed like the more CPUs, the higher the latecy. I started out with 3 cpus. (2 hyperthread-paired, plus one more.) Dropping down to one hyperthread pair helped, and dropping down to one VCPU helped the most.  (All yellow, some green.) Audio "crackling" through my USB headset was also better, but still slightly noticable.

 

Passing through the USB controller solved almost all the audio problems. Boot time was also noticeably faster on my SSD, without the spinner pausing. (10 seconds to boot instead of 30.) DPC latency started as green/yellow, with spikes up to red.

 

With this setup, I was able to play Overwatch and it seemed as good as bare metal.

 

I didn't bother passing through both HDMI video and audio. It didn't seem to matter.

 

Launching a VM that runs on other cores caused intermittent red. This made me think that KVM emulation was stealing CPU cycles, so I isolated the CPUs. isolcpus=1,2,3,5,6,7 made it much worse -- ~8000 microseconds. Pairing CPUs didn't help. I wonder if only leaving 2 cores for unraid causes the usb device sharing to have a harder time?

 

After reverting isolcpus, I noticed that if I play an mpg, the latency drops to under 1000 microseconds. If I stop it, latency spikes up to 4000 microseconds. Maybe that's the performance management of the CPU from the host? I didn't tweak anything with the "Tips and Tweaks" plugin -- so my scaling governor wasn't set for "performance". I've since changed that, but I haven't re-verified this behavior.

 

I've also pushed my luck and installed a 2nd GPU, replacing my Supermicro MV8 controller with a cheapo PCI Express 2.0 x1 one. My DPC delays are back into all-red territory, so I have some more experimentation to do. I'm suspicious of the high qemu host CPU usage when the VM is idle. Basically this is eating up 2 cores. I've seen some things online that indicate that Windows interrupt processing could be causing problems, which would coincide with the crackling audio and high DPC delays that I'm seeing again with this setup. For folks with more cores than me, pushing the KVM workload to isolated cores might be working for you, but I don't have enough cores to do that. :(

 

I'm working with half-baked theories here, so any input is welcome...

Link to comment

Okay, I tried a number of things, and now I can play 2 games at a time with 3 CPUs each. I added the extra vCPU because Overwatch kinda has trouble with just 2 cores. No audio crackling, but occasionally the audio drops for about 5 seconds. Graphics aren't perfect, but playable.

 

I did a number of things, and am not sure what might have helped. DPC delays are still high. CPU load is 80% total for 3 VMs, versus maybe 15% total in the VMs. Before I started fiddling the total host CPU was around 150%. These aren't very scientific numbers, BTW.

 

- Switched clocks in all my VMs. bcdedit /set {default} USEPLATFORMCLOCK on.

- Enable hyper-v in all the VMs, including Windows 7.

- Uninstalled VirtIO Balloon driver, since I don't plan on overprovisioning the memory. I figure it's just going to generate interrupts...

 

Does anyone know when it's safe to uninstall the Serial driver?

Link to comment
  • 3 weeks later...

EDIT: Nothing changed with disabling Hyper Threading! :(

 

Same here. I turned down some settings in Overwatch and now I can play 2 VMs at the same time, at an average of ~40 FPS. Very rarely the FPS will drop down to ~20, or things "lock up" for 5 seconds. It's good enough.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.