Minimizing Latency Help


Recommended Posts

Hi guys,

 

I'm still new to Unraid and especially Linux so please forgive :)

I am now on Unraid 6.4 rc6. But the issue was still on 6.3.5.

I've attached my latencymon and dpc latency checker graphs.

My Win10 XML and syslinux file are there also.

I noticed when setting the XML for the VM, when I chose 1,2,3,7,8,9 the XML didn't look to reflect that. Unsure if the that's the problem.

At the time the graphs were taken, the dockers were all off with no programs open in the VM.

I've tried a few things but they aren't improving, after ready many threads today I thought I have to post my own.

I'm using my VM for a lot of things, however I need an optimal gaming experience for Competitive play.

 

Specs:

5930K - stock freq under liquid 

EVGA x99 Classified - latest bios update, all power saving off.

32GB DDR4

GTX1070 - passed through with ROM


Current dockers on 24/7:

duckdns --cpuset-cpus=4,10

hydra --cpuset-cpus=4,10

letsencrypt --cpuset-cpus=4,10

netdata

nextcloud --cpuset-cpus=4,10

PlexMediaServer --cpuset-cpus=5,11

sonarr --cpuset-cpus=4,10

Subsonic --cpuset-cpus=4,10

 

 

My thread parings:

cpu 0 <===> cpu 6 Pinned Emulator in VM
cpu 1 <===> cpu 7 VM
cpu 2 <===> cpu 8 VM
cpu 3 <===> cpu 9 VM
cpu 4 <===> cpu 10 Dockers
cpu 5 <===> cpu 11 Plex

Tweaks:

Disabled NIC Flow Control
Disabled NIC Offload 
Disk Cache 'vm.dirty_background_ratio' 2%
Disk Cache 'vm.dirty_ratio' 3%
CPU Scaling Governor Performance
Enabled Intel Turbo Boost
 
 

Hopefully that's all the info you need to give me a hand.

Spent the last 6 hours trying to figure this out :(

 

Thank you!

 

 

latencymon.png

dpc.png

win10xml.txt

syslinux.txt

Edited by marshy919
Link to comment

Got it down to just yellow bars in DPC Latency checker.

Not sure exactly what I helped but I done the following.

 

Leatrix Latency Fix

TCPIP Optimizer (optimal settings)

Changed in device manager the link from 10G to 1G.

 

Seems significantly smoother.

I also finally understood the vcpu and cpuset meaning :)

Link to comment

Here's my latency from a Windows 7 VM:

 

I have thread pairs 6,7, 14 and 15 isolated in my flash config and then my XML cpu config is thus:

 

  <vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='7'/>
    <vcpupin vcpu='1' cpuset='14'/>
    <vcpupin vcpu='2' cpuset='15'/>
    <emulatorpin cpuset='6'/>
  </cputune>

 

I found that if I didn't allocated the emulatorpin to an isolated core I would have lots of spikes of red on the latency checker. If I use the isolated cores for VM allocation and emulatorpin allocation I get pretty much green across the board as per the screenshot.

latency.jpg

Link to comment

I just tried the above:

  append isolcpus=0,1,2,3,6,7,8,9 initrd=/bzroot (added 0,6).

    <emulatorpin cpuset='0,6'/> (added to my vm XML)

 

Seems to be very unstable latency though.

Each reboot gives a different result without changing anything. 

The 1000 mark seems to be where I am at 99% of the time though.
 

Untitled.png

Link to comment
9 hours ago, allanp81 said:

Try to avoid using the lower cores as apparently unraid will always use those.

Even if they are isolated?

I've now isolated 10 out of 12 possible.

You think it would help if I had the pair for unraid tasks? Eg. 0,6

Move the emulatorpin to 1,7.

VM allocate the rest?

 

 

 

Link to comment

I followed your XML suggestion above.

But changed it to suit my 6 core setup.

Latency is sitting around 600 stable.

I see yours is still a lot lower, which hardware you using?

Can that be the cause?

My 5930K is currently OCd to 4.3Ghz, haven't affected latency either way.

 

  <vcpu placement='static'>3</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='4'/>
    <vcpupin vcpu='1' cpuset='9'/>
    <vcpupin vcpu='2' cpuset='10'/>
    <emulatorpin cpuset='3'/>
  </cputune>

 

  append isolcpus=3,4,9,10

 

Ran overnight with all the drivers responding in less then 1ms.

Only 1 caused an issue whilst typing this, the ndis.sys driver. 

That went up to 1.15, still unnoticeable.

 

Tonight i'll try adding more cores, as I need more FPS than 3 cores will provide.

 

Thanks

Edited by marshy919
Link to comment

I'm running a Xeon E5-2620 v4 which is an 8 core/16 thread CPU. I'm running this on an Asrock Extreme 4 motherboard which uses the X99 chipset.

 

I'm also using a Geforce 710 passed through to the VM as well as a PCI-E nic passed through.

Edited by allanp81
Link to comment

Added more cores to the VM and flash.

Only made it worse :(

8GB ram down from 16GB didn't change.

No dockers running and minimal plugins.

 

    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='9'/>
    <vcpupin vcpu='6' cpuset='10'/>
    <vcpupin vcpu='7' cpuset='11'/>
    <emulatorpin cpuset='2'/>

 

  append isolcpus=2,3,4,5,8,9,10,11

 

Checked with LatencyMon what is causing the issue.

USBPORT.sys 28ms

storport.sys 28ms

ndis.sys 28ms

nvlddmkm.sys 16ms

 

Untitled.png

Edited by marshy919
Link to comment

This is good now, still not as low as I want, but it's not spiking at all when running all my dockers.

Seems like all my dockers and unraid can run together with only 2 cores.

Might OC to 4Ghz to give it a bit more boost.

Stable 120fps in OW too, very low CPU usage with 10 cores.

 

    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='7'/>
    <vcpupin vcpu='5' cpuset='8'/>
    <vcpupin vcpu='6' cpuset='9'/>
    <vcpupin vcpu='7' cpuset='10'/>
    <vcpupin vcpu='8' cpuset='11'/>
    <emulatorpin cpuset='3'/>

 

  append isolcpus=1,2,3,4,5,7,8,9,10,11 initrd=/bzroot
 

Link to comment

What I noticed on my machine is that I must use paired cores.

 

For instance, I had isolated 6-14 and 7-15, which were the logical threads shown on the dashboard. I assigned 6 to the emulato rpin and the other 3 to the Windows 7 VM. Doing this basically gives me very low latency of around 100-150 and hardly ever spikes above that.

 

I just tried changing this so that only 7,14 and 15 were isolated. With 14 used as the emulator pin and 7-15 assigned to the VM, I jget massive latency spikes that basically make it unusable.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.