Get Fancy with Docker and CPU Pinning

[dimes007]

My new unraid box is a refurb Lenovo D30 Workstation.  It has:

Dual Xeon E5-2670

(and 800gb ssd, 128gb of ram.  Not a ton of drive bays but for ~$1350 I'm so stoked!!  but I digress)

 

There are resources shared (L2 Cache?) between the two logical cpus (hyperthreading) per core.  Does it make sense to pair these "sibling" logical cpus when allocating resources to docker images or vms?

 

On my unraid dashboard these siblings are paired under cpu speed:

"core 0 / 16 1589 MHz         1199 MHz"

"core 1 / 17 1465 MHz  1499 MHz"

Not sure if this pairing is intentional to show they're siblings or was just done to make the display nice. 

 

To me, it makes sense to pair the sibling cpus for a given workload.  So for a 4 logical CPU VM I'd take cores 14, 30, 15, 31.

 

Thanks.

 

[isvein]

Can you see spesific what core an docker uses or do you see it just by the core load?

[Squid]

Can you see spesific what core an docker uses or do you see it just by the core load?

CA - Go to the resource monitor.  Install cAdvisor if its not already installed (a direct link is in resource monitor), then click the icon for the app in question, and it will show you the utlization per core for that app.

 

By default every docker app will use all cores available to unRaid.

[xxredxpandaxx]

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

[aptalca]

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

I have them comma separated as

--cpuset-cpus=0,1,2

[Squid]

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

What does cAdvisor show for the App?  You can install it via CA resource monitor.  It works for me exactly how you have it listed.  Also by 4 cores do you mean 2 hyperthreaded cores?

 

Sent from my SM-T560NU using Tapatalk

 

 

[xxredxpandaxx]

I have them comma separated as

--cpuset-cpus=0,1,2

 

I tried it that way also and it didn't do anything different.

 

What does cAdvisor show for the App?  You can install it via CA resource monitor.  It works for me exactly how you have it listed.  Also by 4 cores do you mean 2 hyperthreaded cores?

 

no, 4 physical cores. CAdvisor shows it is limited to 2 cores (just changed it to two cores to see if it made a difference) and it only shows two cores being used for the sabnzbd on the graph. but I am looking at netdata and it shows all 4 cores being used at 100% and when I stop the sabnzbd container my cpu usage goes back down to around 10%

 

Here is a picture of all 4 cores from when sabnzbd isn't running to when it is. it looks like it starts to use only the first two cores but soon all of the cores seem to be being used.

[BRiT]

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

 

Try switching to NZBGet. 

[xxredxpandaxx]

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

 

Try switching to NZBGet. 

 

I assume it uses the same 100% cpu usage while unpacking so it wouldn't really help to switch to it. I would rather just get the sabnzbd to only use 2 or 3 cores so my server isn't useless while it is unpacking something.

[xxredxpandaxx]

I just tried the same thing with nzbget and it does the same thing, uses all cores and 100% cpu.

[BRiT]

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

[Squid]

It could also be shfs (unraid) spiking the cpu to actually rw the files.

 

Sent from my LG-D852 using Tapatalk

 

 

[xxredxpandaxx]

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores of the CPU.

 

 

It could also be shfs (unraid) spiking the cpu to actually rw the files.

 

Sent from my LG-D852 using Tapatalk

 

I can't imagine that is the case, it shouldn't take that much CPU. Do you know of any way that I can test this though? 

 

Also does anyone have their container working to where it doesn't use 100% CPU while unpacking with sab? Or nzbget?

 

 

Sent from my iPhone using Tapatalk

[Squid]

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores of the CPU.

 

Agree

Also does anyone have their container working to where it doesn't use 100% CPU while unpacking with sab? Or nzbget?

Don't particularly pay that much attention to it.  But I can tell you that the --cpuset-cpus does work.  I test it with [email protected]

 

You'd really have to look at running htop or something so you can see exactly what the top processes are and where they are from.

[BRiT]

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores

 

Right, but try setting it anyways just to ensure even without cpuset that REPAIR doesnt take up more than you want it to.

[sixarm]

For unRAID 6.2, I don't see an option for changing the docker command for a container.  How do I provide  --cpuset-cpus args?

[Squid]

For unRAID 6.2, I don't see an option for changing the docker command for a container.  How do I provide  --cpuset-cpus args?

Advanced view, extra parameters

[blahblah0385]

Did this change for unRAID release 6.3? when i put that in the extra parameters and hit apply it removes the docker and i have to reinstall it.

[-Daedalus]

I could do with a little help here. Not sure I'm understanding this...

 

Main issue: With Plex docker transcoding multiple streams, Minecraft servers (in MineOS docker) experience lag. I'm hoping to reduce this by giving priority to MineOS, but still allowing Plex to use up to all remaining resources.

 

As far as I get it, pinning a docker app to a given core means that anything used by that app that's in the CPU isn't moved out to RAM unless it has to be.

 

So, would this mean I'd pin MineOS (the highest priority in this scenario) to, say, cores 0 and 1, and let Plex run as normal? Does this still allow Plex to use any untapped resources on 0 and 1, or have I just restricted it to the other six cores?

[Squid]

I could do with a little help here. Not sure I'm understanding this...

 

Main issue: With Plex docker transcoding multiple streams, Minecraft servers (in MineOS docker) experience lag. I'm hoping to reduce this by giving priority to MineOS, but still allowing Plex to use up to all remaining resources.

 

As far as I get it, pinning a docker app to a given core means that anything used by that app that's in the CPU isn't moved out to RAM unless it has to be.

 

So, would this mean I'd pin MineOS (the highest priority in this scenario) to, say, cores 0 and 1, and let Plex run as normal? Does this still allow Plex to use any untapped resources on 0 and 1, or have I just restricted it to the other six cores?

In this case, Plex would have access to all of the cores, and MineOS only to cores 0 & 1

 

But, that doesn't necessarily mean that you're going to solve your problem.  What you really want to do here is prioritize one app over another.  And for that, you're going to be playing around with --cpu-shares  (And you can combine cpu shares with cpu pinning)

 

See this FAQ entry  http://lime-technology.com/forum/index.php?topic=40937.msg492111#msg492111

[-Daedalus]

Thanks very much for that.

 

CPU shares: If I were doing things correctly, all my dockers should be at, say 256, and I could increase or decrease that number, depending on priority. (MineOS on 1024, Plex on 512, and the rest on 256).

 

CPU pinning: After reading the OP, I thought it was something a bit different from what it is.So CPU pinning is just "Use only these assigned cores and no more" with other dockers free to use any unused resources. But this is good because nothing from pinned docker is ever paged to RAM.

 

As it stands, all dockers are shared at the default 1024, with Plex at 512, and MineOS is pinned to cores 0,1,2,3 (half of available). Seems to work reasonably well.

[SpaceInvaderOne]

Hi Guys I have made some videos about server tuning that cover Docker container resource management.

 

[bedpan]

Might be worth a new post but I am lost..

 

I updated my plex container with 

--cpuset=0,2,4,6

 

Missed that it should have been --cpuset-cpus

 

So I clicked apply. It did its thing and the docker restarted. Then it was gone. It no longer appears in my list of Dockers. Just gone!

 

Any thoughts? How do I go about finding the config file?

 

Sorry guys.. I really don't know what I am doing. I pretend well though! I needs my plex!

 

edit:

 

I clicked advanced view and saw: 

 

(orphan image)

Image ID: 9b60aa70ec67

linuxserver/plex:latest

 

 

Edited August 22, 2017 by bedpan

[DZMM]

I've just implemented this for the first time and I'm getting errors from fix common applications:

 

Quote

Adding extra parameters by including them in the repository (eg: --cpuset-cpus) is deprecated, and may impact the ability of dockerMan to correctly manage your application (eg: unable to update, etc). The proper way to add extra parameters is through the Extra Parameters section when you edit the container (you will have to hit Advanced Settings) Fix this here: 

should this go in extra parameters now?

 

Thanks

 

UPDATE: found the answer - first post probably needs updating as I didn't realise I had to add the HT pair

 

Quote

 

Add this to the extra parameters section when you add / edit the app:

 

--cpuset-cpus=0,1

This will pin the application to cores 0 & 1  * Note that none of my CPU's have hyperthreading, so I have to assume that you would set the cores the same way as pinning a VM on a hyperthreaded cpu

 

 

 

Edited August 23, 2017 by DZMM
adding answer to help others

[SpaceInvaderOne]

12 hours ago, bedpan said:

Might be worth a new post but I am lost..

 

I updated my plex container with 

--cpuset=0,2,4,6

 

Missed that it should have been --cpuset-cpus

 

So I clicked apply. It did its thing and the docker restarted. Then it was gone. It no longer appears in my list of Dockers. Just gone!

 

Any thoughts? How do I go about finding the config file?

 

Sorry guys.. I really don't know what I am doing. I pretend well though! I needs my plex!

 

edit:

 

I clicked advanced view and saw: 

 

(orphan image)

Image ID: 9b60aa70ec67

linuxserver/plex:latest

 

 

@bedpan. You dont need to worry about the orphaned image it just means there is no docker template for that image.

But to remove it just click it then click remove.

 

This will remove the image yet leave your appdata complete so dont worry about that.

Now to restore the docker container and all its settings we just have to goto the docker tab and click 

add container.

 

then next to templates and you should see the old template saying my-plex etc and it will put back in the template 

and you can then edit it correctly.