Show Me Your Pins! (CPU-Pinning, not 'nterest)

[bungee91]

This isn't too heavily discussed, but there are a couple of threads here and there with some information.

 

I'm interested to know how everyone is allocating CPU's with various VM's, Docker, etc...

My main interest is users who have multiple VM's with a GPU assigned, since this is likely the place you would easily notice lag/stutter/glitches, etc... that could be related to shared resources between processes.

 

However it also makes a lot of sense to pin 4-6 cores (out of 8 ) to all 3 VM's, that way if one needs it, it gets it's work done faster.

Plus I very rarely interact with more than 1 VM at any one time (so watching Tv in 2 locations is unlikely) it could happen, sure, but if the VM had the extra resources available, the one I am using would therefore be quicker.

 

My story

----------------

I have a 4790S so I have a total of 8 vcpu's (4 of these are hyperthreaded cores, no clue how that works between "real" and "hyperthreaded" ones).

 

I have no set pins for Docker, they are as is (which should mean all are available by default).

My Dockers I run regularly are:

MythTv

CouchPotato

MadSonic

MariaDB

Kodi-Headless

 

I have 3 Windows VM's, 2 with GPU's assigned.

Windows 8.1 - Headless, Torrents, 2vcpu's (6,7)

Windows 10 (Kitchen)- GT720, HTPC, 2vcpu's (4,5)

Windows 10 (Living Room) - GT720, HTPC 2vcpu's (2,3)

 

I am thinking about this more as every once in a while I get a weird playback error on the kitchen Tv, gets all slow and awful like I hit slow motion (I'm melting....mmmmeeeeeelllllttttttiiiinnnnnngggg)... 

Hit stop, play, all is well.

 

I believe that it is possible that an interrupt or resource assignment causes this issue.

I was running the stable virtio drivers, just updated to the latest .109, will see if that helps.

 

I just made the change to enable MSI interrupts in Windows for this GPU thinking this may solve my issue (surprised it wasn't enabled by default), however haven't had time to test yet.

 

[methanoid]

So what happened? I've been trawling forum for info on how to set up CPU pins.... Similar to you... 4 core, 4 HT... Some Dockers inc Plex ... Win10VM for games - Wondering what I can assign to what or if I need to mess with them? Could I run two GPU-assisted VMs?

 

[bungee91]

Well, a lot has changed since then (I now have 6 cores, 12 threads!).

I never did get any real information on the topic, however the MSI interrupt was the cause for the "demonic audio" issue I was having in the 1 VM.

You should have no issue running 2 VM's with GPU's assigned.

I don't think you quite have the horsepower to have them both doing heavy gaming, but certainly for 1 of them at a time shouldn't be an issue, plus the other stuff you mentioned.

 

I upgraded to an Intel Extreme 5930k mainly as I got tired of "workarounds" using ACS downstream to allow devices in their own IOMMU grouping.

This may not be an issue for you (more motherboard dependent), however the "Extreme" versions support ACS on root ports, and sure enough I have proper isolation between all of my devices now!

As for pinning, I try to have cores exclusive to each VM, and also pin specific cores to dockers.

With 8 cores you could pin the last 4 (4-7) to your gaming VM, 2 to all of your dockers (0-1) and the other 2 to you other VM (2-3).

I think the more important part of this is a minimum of 4 cores for your gaming rig, the rest you can experiment with.

You do use Plex however (I do not), so you may want to pin that to specific cores other than your gaming VM as it can require higher CPU allocation at times.

 

[Maniac]

Hmmm, I am also wondering how my gaming VM is supposed to be setup. I have a i5 6600K "Skylake" with 4 cores.

 

I am currently running:

Dockers: Sabnzbd, Deluge, WebGrab++ (these don't do anything most of the time).

OpenElec VM (allways on - used for watching liveTV - Assigned 1 core and 1gb RAM)

 

So I am looking to add a Win10 VM for gaming. It will be powered on only when I want to game. Is it safe to assign all 4 cores to this one?

[methanoid]

Would be nice to hear from LT on this. I'm also upgrading to an 8C/16T CPU and S2011 but I'd like to know the answers:

 

1)How are CPU assignments made and how should they be?

 

2)Can you assign the same CPU to multiple VMs? Does it then split CPU time (they fight it out)? Or does that cause major problems

 

3) Can you protect CPUs for unRAID operation (ie block them completely from VM contention) to keep responsiveness of unRAID services (eg. Plex or SMB transfers). Or is the only way by making sure you don't assign certain CPUs to VMs?

 

4) For machines with Hyperthreading is there an easy way to identify the HT cores (as they are not gonna have the same power as a normal core)? I ASSUME the even cores are real and the odd cores the HT cores but I'd like to know.

 

[bungee91]

Would be nice to hear from LT on this. I'm also upgrading to an 8C/16T CPU and S2011 but I'd like to know the answers:

 

1)How are CPU assignments made and how should they be?

 

2)Can you assign the same CPU to multiple VMs? Does it then split CPU time (they fight it out)? Or does that cause major problems

 

3) Can you protect CPUs for unRAID operation (ie block them completely from VM contention) to keep responsiveness of unRAID services (eg. Plex or SMB transfers). Or is the only way by making sure you don't assign certain CPUs to VMs?

 

4) For machines with Hyperthreading is there an easy way to identify the HT cores (as they are not gonna have the same power as a normal core)? I ASSUME the even cores are real and the odd cores the HT cores but I'd like to know.

 

3) You can do the opposite if you want to remove UnRAID from using cores by adding this in your syslinux.cfg between this "append initrd=/bzroot"

isolcpus=(insert cores here, separated by a ,) 

Put in the cores you want to isolate, and UnRAID won't use them for its own uses (but you can assign them to a VM).

I'm not sure why you'd need the opposite really, just don't assign them and you should be good.

You can also put in parameters to each docker to use specific cores if you know you're not using a particular core, and want to make it exclusive to that docker.

Use

--cpuset-cpus="(insert cores here, separated by a ,)"

 

4) Yes, as JonP talked about in another thread there is a script to check latencies between cores to help distinguish which are in the same logical core.

https://github.com/awilliam/cpu-latencies

This does not run natively in SSH for UnRAID, I assume netperf needs to be installed or something of that nature.

Talk of it https://www.redhat.com/archives/vfio-users/2015-September/msg00041.html

https://www.redhat.com/archives/vfio-users/2015-September/msg00175.html