Docker still scheduling on isolated CPU cores/threads

[dcoulson]

I have a 24C/48T system which has "isolcpus=6-11,30-35" set at boot time to isolate 6C/12T for a VM. I am still seeing docker containers running on these cores when I do "ps -eo pid,psr,comm", and see their CPU utilization in the dashboard with the VM shut down.

 

Do I have to explicitly pin the containers to other cores/threads, or is this functionality broken? I don't see a way to set the cpuset globally for the docker cgroup, and since I have 60+ containers I was hoping to not have to manually set the cpuset on each one - that's a lot of clicking.

 

Is there a better approach for this, or am I misunderstanding how isolcpu works?

[Squid]

Shouldn't have to pin the containers.  Only reason OTOH a container would wind up running on an isolated core is if you've actually pinned the container to that isolated core.

[dgstewart]

I've noticed this behavior too, despite the last 2core/2threads of my system being isolated, one docker container in particular will exclusively sit on an isolated thread and keep it pinned at 100%.  No amount of tinkering with privileged/unprivileged container settings has changed this.  I have confirmed that the docker is not assigned to any cores/threads and is most certainly not assigned to these isolated cores.

Edited January 26, 2023 by dgstewart

[aarontry]

I can confirm too. Isolated 2 cores/4 threads but Tdarr node is still using the isolated cores. Potentially a bug.

[dgstewart]

For what it's worth, I have not observed this behavior since I changed my pinned cores from the last two cores of my system (CPU 22 - HT 46, CPU 23 - HT 47)

to the first few cores of my system (CPU 1 - HT 25, CPU 2 - HT 26).
Not sure if this is still the case, or ever was, but I remember being told specifically not to isolate Core 0 since the Unraid Host OS uses this for its related functions.

To reiterate, my particular issue was that a NON-pinned and NON-privileged container, after some time of running normally, was getting assigned to the last thread on my system (HT 47) I can only speculate as to why this was happening.

Hopefully this helps somebody and maybe confirms my procedure as a temporary solution.

[aarontry]

I have a dual CPU system and I have the 2nd (CPU 1 - HT 29) and 3rd (CPU 2 - HT 30) on the first socket isolated for TrueNAS exclusively. But I have observed Tdarr using CPU 1 for video transcoding. Any solutions?