Get Fancy with Docker and CPU Pinning


jonp

Recommended Posts

13 minutes ago, DZMM said:

I've just implemented this for the first time and I'm getting errors from fix common applications:

 

should this go in extra parameters now?

 

Thanks

 

UPDATE: found the answer - first post probably needs updating as I didn't realise I had to add the HT pair

 

 

 

@DZMM yes add that to extra parameters now.

I did a video a while back if want to look at it you will see at video 2 at 4minutes 10 seconds that part :)

 

Link to comment

Awesome reply.. Thanks Grid for taking the time. I did open another thread in general support (got chastised for it as well ;-) ). Anyways I did delete the orphan and added Plex again. Took me sometime to get my shares right but all is good now. 

I went back to customizing the CPU's again. I need to read better I guess.. I did get it working in the end though after I realized you have to add the flags to the extra parameters.. All good now and I think my plex is working better. I was having some problems with a high use VM and plex intermingling and causing plex to bog down during live transcoding. Well I think thats what was going on. I will need to run it for a few days to see if its actually fixed.

Again thanks folks!

Quote

 

@bedpan. You dont need to worry about the orphaned image it just means there is no docker template for that image.

But to remove it just click it then click remove.

 

 

Edited by bedpan
Link to comment
  • 3 months later...
On 8/23/2017 at 11:46 AM, bonienl said:

Is there a need / wish to do cpu selection for Dockers similar to what is done for VMs?

 

Did this result in anything?

 

I just tried assigning cpus to dockers that are isolated cpus but it won't work. If I assign cpu 20-29 to a docker and those threads are isolated in the syslinux, it only uses cpu 20. As soon as I remove isolcpus the docker gets the specified 20-29.

Link to comment
8 minutes ago, bonienl said:

 

Not yet. 6.4 is close to release, but it will be something for 6.5.

Spinning this a bit further... The only thing I miss, coming from ESXi, except snapshots is vspheres awesomeness when it comes to handling cpu resources. Now I don't have any expectations like that of course but is there any planned work on improved automated cpu handling for virtualized systems, with numa etc, for future releases?

Link to comment
  • 2 months later...
  • 2 months later...
  • 2 weeks later...

Seems that the CPU numbering in unraid isnt the same as the extra setting for syntching CPU pinning. Anyone else has the same issue?

I gave 4 vCPUs that are different from the VMs but seems to overlap on htop! 

For example I did --cpuset-cpus=2,46,3,47 and my VMs had completely different sets but if you htop on the unraid you will see docker competing for the save 'core/multithread'. 

Link to comment
  • 3 weeks later...
On 8/23/2017 at 10:46 AM, bonienl said:

Is there a need / wish to do cpu selection for Dockers similar to what is done for VMs?

 

 

I was searching the forums trying to find how to do this as I have a docker (sab) that's taking too much CPU away from other applications (emby), so wanted to pin the 2 dockers in question to fixed unique cores.

Tick boxes would be a very nice feature to quickly assign cores to dockers. And maybe also a way to list all dockers and their tick box core assignment in a big list so assigning cores to all dockers at the same time is easy (won't need a pen and paper to remember which docker (or VM) uses what core, etc.) including assigning cores to unRAID.

  • Like 1
Link to comment
  • 2 weeks later...

Posted this in general support, but it doesn't seem to be getting much traction:

 

 

If you have your syslinux configured like mine, and have isolated some cores from unRAID, like so:

label unRAID OS
  menu default
  kernel /bzimage
  append isolcpus=8-15,24-31 initrd=/bzroot

Any time I pin any of these threads to a Docker container using:

--cpu-setcpus="14-15,30-31"

That container is limited to only using the first thread.

 

VMs don't seem to be affected by this, only Docker stuff.

Running a 1950X. Present in 6.5.2/3.

 

Can anyone else reproduce this?

Edited by -Daedalus
Link to comment
  • 1 month later...

Can I combine --cpuset= with append isolcpus= to block unraid from using certain cores but allow certain dockers to use those cores e.g. could I do the following at the same time:

 

# stop unraid using the cores assigned to my VM
append isolcpus= 8,9,10,22,23,24

#within VM xml assign the same cores i.e. VM can use

  <cputune>
    <vcpupin vcpu='0' cpuset='8'/>
    <vcpupin vcpu='1' cpuset='22'/>
    <vcpupin vcpu='2' cpuset='9'/>
    <vcpupin vcpu='3' cpuset='23'/>
    <vcpupin vcpu='4' cpuset='10'/>
    <vcpupin vcpu='5' cpuset='24'/>
    <emulatorpin cpuset='0,14'/>
  </cputune>

# allow priority dockers e.g. Plex to access all cores

--cpuset-cpus=0-27

at the moment I stop most dockers using cores 8,.9,10,22,23,24 but I'm wondering if I can also stop unraid, but let other dockers through.


Thanks

 

Edit: If I do append isolcpus=8,9,10,22,23,24 does this automatically exclude dockers from using these cores unless I do --cpuset-cpus=0-27 in the docker settings??

Edited by DZMM
Link to comment
12 minutes ago, BRiT said:

No, IIRC. isolcpus blocks the cores from being used by the host, where the dockers are run at.

 

Pity, that would have been cool.  My plex usage can get high at times, so I'm nervous about not giving it access to all my cores but I really want to try and fix audio lag in my W10 VM.  I think I'll doing isolcpus as a test to see if it solves the problem first.

 

Thanks

Link to comment
  • 2 years later...

Quick question regarding CPU pinning and Docker parameters,

 

A few months ago, I found the docker parameter "  --cpu-shares=2 " . This, to my understanding, limits the docker from using more than 50% of the total CPU. It helped when I had a folding docker that liked to bring my whole system to a halt. (boinc seems to be the better option that what I was using at the time.)

 

However, when using the CPU pinning to limit cores a Docker has access to, would this command half that? such as if I pinned 4 cores, and used the aforementioned parameter, does that limit the docker to have only 2 cores?

 

please let me know your thoughts and if I'm understanding this correctly. 

Link to comment
20 hours ago, Aerodb said:

I found the docker parameter "  --cpu-shares=2 " . This, to my understanding, limits the docker from using more than 50% of the total CPU. It helped when I had a folding docker that liked to bring my whole system to a halt. (boinc seems to be the better option that what I was using at the time.)

It sets the CPU usage of the container to be the absolute lowest priority if another container wants the cycles.  If nothing else wants it then it will run and use as many cycles as it wants.

 

If combined with pinning, then basically the same thing happens, but it's confined onto the pinned cores.

  • Thanks 1
Link to comment
1 hour ago, Squid said:

It sets the CPU usage of the container to be the absolute lowest priority if another container wants the cycles.  If nothing else wants it then it will run and use as many cycles as it wants.

 

If combined with pinning, then basically the same thing happens, but it's confined onto the pinned cores.

That's very helpful. Thank you. Are there other priorities? Such as, can I end the command with a "1" to have a higher priority? or "0" for the highest priority?

Link to comment
  • 10 months later...

Just reading on cpu pinning, I guess there's a use case for this and not for everyone? I ran many game servers and dockers. I have a Xeon Gold with 14 cores 28 threads. My overall load is rarely over 20%. I never had any performance issue, I guess unless this start to happen, no need to pin anything?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.