Get Fancy with Docker and CPU Pinning


jonp

Recommended Posts

My new unraid box is a refurb Lenovo D30 Workstation.  It has:

Dual Xeon E5-2670

(and 800gb ssd, 128gb of ram.  Not a ton of drive bays but for ~$1350 I'm so stoked!!  but I digress)

 

There are resources shared (L2 Cache?) between the two logical cpus (hyperthreading) per core.  Does it make sense to pair these "sibling" logical cpus when allocating resources to docker images or vms?

 

On my unraid dashboard these siblings are paired under cpu speed:

"core 0 / 16 1589 MHz         1199 MHz"

"core 1 / 17 1465 MHz  1499 MHz"

Not sure if this pairing is intentional to show they're siblings or was just done to make the display nice. 

 

To me, it makes sense to pair the sibling cpus for a given workload.  So for a 4 logical CPU VM I'd take cores 14, 30, 15, 31.

 

Thanks.

 

Link to comment
  • 2 months later...

Can you see spesific what core an docker uses or do you see it just by the core load?

CA - Go to the resource monitor.  Install cAdvisor if its not already installed (a direct link is in resource monitor), then click the icon for the app in question, and it will show you the utlization per core for that app.

 

By default every docker app will use all cores available to unRaid.

Link to comment
  • 1 month later...

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

What does cAdvisor show for the App?  You can install it via CA resource monitor.  It works for me exactly how you have it listed.  Also by 4 cores do you mean 2 hyperthreaded cores?

 

Sent from my SM-T560NU using Tapatalk

 

 

Link to comment

I have them comma separated as

--cpuset-cpus=0,1,2

 

I tried it that way also and it didn't do anything different.

 

What does cAdvisor show for the App?  You can install it via CA resource monitor.  It works for me exactly how you have it listed.  Also by 4 cores do you mean 2 hyperthreaded cores?

 

no, 4 physical cores. CAdvisor shows it is limited to 2 cores (just changed it to two cores to see if it made a difference) and it only shows two cores being used for the sabnzbd on the graph. but I am looking at netdata and it shows all 4 cores being used at 100% and when I stop the sabnzbd container my cpu usage goes back down to around 10%

 

Here is a picture of all 4 cores from when sabnzbd isn't running to when it is. it looks like it starts to use only the first two cores but soon all of the cores seem to be being used.

cpuPIN.PNG

Link to comment

Put this in the extra parameters area and sab still uses all cores on my cpu.

 

--cpuset-cpus=0-2

 

am I missing something? its a quad core cpu btw.

 

Try switching to NZBGet.  8)

 

I assume it uses the same 100% cpu usage while unpacking so it wouldn't really help to switch to it. I would rather just get the sabnzbd to only use 2 or 3 cores so my server isn't useless while it is unpacking something.

Link to comment

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores of the CPU.

 

 

It could also be shfs (unraid) spiking the cpu to actually rw the files.

 

Sent from my LG-D852 using Tapatalk

 

I can't imagine that is the case, it shouldn't take that much CPU. Do you know of any way that I can test this though? 

 

Also does anyone have their container working to where it doesn't use 100% CPU while unpacking with sab? Or nzbget?

 

 

Sent from my iPhone using Tapatalk

Link to comment

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores of the CPU.

 

Agree

Also does anyone have their container working to where it doesn't use 100% CPU while unpacking with sab? Or nzbget?

Don't particularly pay that much attention to it.  But I can tell you that the --cpuset-cpus does work.  I test it with Folding@home

 

You'd really have to look at running htop or something so you can see exactly what the top processes are and where they are from.

Link to comment

I dont know if theres params for extraction, but from memory there are params for PAR with NZBGet. Have you configured them as to how many threads it should use? It might be part of the extra params for PAR.

 

This shouldn't matter because with the --cpuset it should not allow the container to use more than two cores

 

Right, but try setting it anyways just to ensure even without cpuset that REPAIR doesnt take up more than you want it to.

Link to comment
  • 4 weeks later...
  • 4 weeks later...
  • 2 weeks later...

I could do with a little help here. Not sure I'm understanding this...

 

Main issue: With Plex docker transcoding multiple streams, Minecraft servers (in MineOS docker) experience lag. I'm hoping to reduce this by giving priority to MineOS, but still allowing Plex to use up to all remaining resources.

 

As far as I get it, pinning a docker app to a given core means that anything used by that app that's in the CPU isn't moved out to RAM unless it has to be.

 

So, would this mean I'd pin MineOS (the highest priority in this scenario) to, say, cores 0 and 1, and let Plex run as normal? Does this still allow Plex to use any untapped resources on 0 and 1, or have I just restricted it to the other six cores?

Link to comment

I could do with a little help here. Not sure I'm understanding this...

 

Main issue: With Plex docker transcoding multiple streams, Minecraft servers (in MineOS docker) experience lag. I'm hoping to reduce this by giving priority to MineOS, but still allowing Plex to use up to all remaining resources.

 

As far as I get it, pinning a docker app to a given core means that anything used by that app that's in the CPU isn't moved out to RAM unless it has to be.

 

So, would this mean I'd pin MineOS (the highest priority in this scenario) to, say, cores 0 and 1, and let Plex run as normal? Does this still allow Plex to use any untapped resources on 0 and 1, or have I just restricted it to the other six cores?

In this case, Plex would have access to all of the cores, and MineOS only to cores 0 & 1

 

But, that doesn't necessarily mean that you're going to solve your problem.  What you really want to do here is prioritize one app over another.  And for that, you're going to be playing around with --cpu-shares  (And you can combine cpu shares with cpu pinning)

 

See this FAQ entry  http://lime-technology.com/forum/index.php?topic=40937.msg492111#msg492111

Link to comment

Thanks very much for that.

 

CPU shares: If I were doing things correctly, all my dockers should be at, say 256, and I could increase or decrease that number, depending on priority. (MineOS on 1024, Plex on 512, and the rest on 256).

 

CPU pinning: After reading the OP, I thought it was something a bit different from what it is.So CPU pinning is just "Use only these assigned cores and no more" with other dockers free to use any unused resources. But this is good because nothing from pinned docker is ever paged to RAM.

 

As it stands, all dockers are shared at the default 1024, with Plex at 512, and MineOS is pinned to cores 0,1,2,3 (half of available). Seems to work reasonably well.

Link to comment
  • 8 months later...
  • 1 month later...

Might be worth a new post but I am lost..

 

I updated my plex container with 

--cpuset=0,2,4,6

 

Missed that it should have been --cpuset-cpus

 

So I clicked apply. It did its thing and the docker restarted. Then it was gone. It no longer appears in my list of Dockers. Just gone!

 

Any thoughts? How do I go about finding the config file?

 

Sorry guys.. I really don't know what I am doing. I pretend well though! I needs my plex!

 

edit:

 

I clicked advanced view and saw: 

 

(orphan image)

Image ID: 9b60aa70ec67
linuxserver/plex:latest

 

 

Edited by bedpan
Link to comment

I've just implemented this for the first time and I'm getting errors from fix common applications:

 

Quote

Adding extra parameters by including them in the repository (eg: --cpuset-cpus) is deprecated, and may impact the ability of dockerMan to correctly manage your application (eg: unable to update, etc). The proper way to add extra parameters is through the Extra Parameters section when you edit the container (you will have to hit Advanced Settings) Fix this here: 

should this go in extra parameters now?

 

Thanks

 

UPDATE: found the answer - first post probably needs updating as I didn't realise I had to add the HT pair

 

Quote

 

Add this to the extra parameters section when you add / edit the app:

 

--cpuset-cpus=0,1

This will pin the application to cores 0 & 1  * Note that none of my CPU's have hyperthreading, so I have to assume that you would set the cores the same way as pinning a VM on a hyperthreaded cpu

 

 

 

Edited by DZMM
adding answer to help others
Link to comment
12 hours ago, bedpan said:

Might be worth a new post but I am lost..

 

I updated my plex container with 


--cpuset=0,2,4,6

 

Missed that it should have been --cpuset-cpus

 

So I clicked apply. It did its thing and the docker restarted. Then it was gone. It no longer appears in my list of Dockers. Just gone!

 

Any thoughts? How do I go about finding the config file?

 

Sorry guys.. I really don't know what I am doing. I pretend well though! I needs my plex!

 

edit:

 

I clicked advanced view and saw: 

 

(orphan image)

Image ID: 9b60aa70ec67
linuxserver/plex:latest

 

 

@bedpan. You dont need to worry about the orphaned image it just means there is no docker template for that image.

But to remove it just click it then click remove.

599d3e537287b_ScreenShot2017-08-23at09_32_49.thumb.png.e3b7e03ea4e933cdfa6021601786a217.png

 

This will remove the image yet leave your appdata complete so dont worry about that.

Now to restore the docker container and all its settings we just have to goto the docker tab and click 

add container.

599d3e56883a6_ScreenShot2017-08-23at09_33_15.thumb.png.bdd10c134c8954ad8690c7a139a5f318.png

 

then next to templates and you should see the old template saying my-plex etc and it will put back in the template 

and you can then edit it correctly.

599d3e58f3148_ScreenShot2017-08-23at09_33_34.thumb.png.ff2929c5acb5721dc6c6efccc9308fe3.png

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.