Docker Image Disk Utilization [SOLVED]


TODDLT

Recommended Posts

I hope this is the correct area for this question.  

 

Every once in a while I get an email notification that says "unRAID Status: Warning - Docker image disk utilization of 88%.  Then later it I get a "returned to normal" email.

 

My docker and app data are stored on a cache disk (SSD) that is only used for appdata.  It's a 250 GB disk and 27% full.

I have attached two images that show my docker settings and memory usage (which shows docker usage at 76%).   

 

Is it possible when my Plex app is transcoding and/or doing DVR recording, and using up too much memory?  If so can this be offloaded to the SSD Cache?

 

I have plenty of cache drive space, is there a setting wrong that is driving it to use memory instead of the cache disk?  how do I get more Docker Image space?

 

 

Thanks for any help.

 

1007865769_MemoryUsage.JPG.ab9ae300e3b89653d9fee53dc629a1d1.JPG428996781_DockerSettings.thumb.JPG.371cb7dfecd30b06eb205ce60248f60a.JPG

Edited by TODDLT
Link to comment

The docker image utilization will grow and shrink normally during container updates. If you don't want the warnings, either expand the image so the temporary growth during updates doesn't pass the warning level, or change the warning level up a couple points until you don't get it during normal updates. If there is too little space, the container update may fail, possibly corrupting the image forcing you to recreate it.

 

What I do is make sure there is enough free space inside the image so normal updates don't cross the warning threshold.

Link to comment
6 hours ago, jonathanm said:

The docker image utilization will grow and shrink normally during container updates. If you don't want the warnings, either expand the image so the temporary growth during updates doesn't pass the warning level, or change the warning level up a couple points until you don't get it during normal updates. If there is too little space, the container update may fail, possibly corrupting the image forcing you to recreate it.

 

What I do is make sure there is enough free space inside the image so normal updates don't cross the warning threshold.

How do you expand the size of the docker image?   Is this housed in RAM, or in Cache?

 

None of my dockers "auto-update" and this warning comes around at seemingly random times.  What "update" would be occurring to cause this?

 

14 hours ago, itimpi said:

You should probably prob9de a screenshot of the settings you have for your Plex docker container if that is the one you suspect might be causing your issue.

I'll provide this when I get home in the hour.

Link to comment
11 minutes ago, TODDLT said:

How do you expand the size of the docker image? 

Settings - docker.  Stop the service, increase the size, restart the service

12 minutes ago, TODDLT said:

Is this housed in RAM, or in Cache?

Wherever you've set it in Settings - Docker.  Usually on the cache drive

15 hours ago, TODDLT said:

Every once in a while I get an email notification that says "unRAID Status: Warning - Docker image disk utilization of 88%.

 What you want to do is go to Docker, and then hit Calculate Container Size when that happens, and before you get 

 

15 hours ago, TODDLT said:

"returned to normal" email.

 

Net result is that some app is storing stuff within the docker image and then it gets moved out / deleted.  Plex is a possibility, but download clients are IMO more common for this.

Link to comment
3 hours ago, Squid said:

Settings - docker.  Stop the service, increase the size, restart the service

Wherever you've set it in Settings - Docker.  Usually on the cache drive

 What you want to do is go to Docker, and then hit Calculate Container Size when that happens, and before you get 

 

 

Net result is that some app is storing stuff within the docker image and then it gets moved out / deleted.  Plex is a possibility, but download clients are IMO more common for this.

Thanks!   It's been long enough since I first enabled Dockers I wasn't remembering.  

The image file was set to 10GB and I upped it to 30GB so now down to 23% usage.

 

There is also something called Log rotation, which I imagine is saving 1, 2, or 3 files in history for troubleshooting purposes.  Mine was set to 1 file 50 MB in size.   I upped it to 3 files 100 MB.  Not sure in relative terms how big this should be with on 7 dockers installed and 3 that run under daily normal situations.  I assume the more dockers you have running the larger that file should be?

 

Doesn't seem particularly important at this point but no download clients installed here.  The only real active Docker is Plex.

 

Thanks again, that should hopefully solve the issue.

Edited by TODDLT
Link to comment
  • 2 years later...
On 10/18/2019 at 2:14 PM, JonathanM said:

The docker image utilization will grow and shrink normally during container updates. If you don't want the warnings, either expand the image so the temporary growth during updates doesn't pass the warning level, or change the warning level up a couple points until you don't get it during normal updates. If there is too little space, the container update may fail, possibly corrupting the image forcing you to recreate it.

 

What I do is make sure there is enough free space inside the image so normal updates don't cross the warning threshold.

 

How do I change the warning levels? I don't see why I should be getting constant warnings just because I'm at 70%.

Link to comment
  • 1 year later...

Excuse my ignorance but I've had a 40gb docker image allocation for a while and not noticed it being an issue, I have now upped this to 100gb as I've been getting warnings it's almost full.. I've recently set up compreface and doubletake alongside frigate. Where do I begin to work out why the docker image is getting so large? I have 38 docker containers deployed, so unsure where I should begin?

Link to comment
  • 1 month later...
On 10/18/2019 at 11:10 PM, Hoopster said:

Generally, you docker.img should never need to be any larger than 20GB.  If it needs to be larger than that, that's usually an indication that some docker container(s) are configured improperly and are storing data in docker.img.

Hi. @Hoopster, does this rule of thumb apply to everyone, or only to those who doesn't use UnRaid as a virtualization server with 25 containers running? 😁

image.png.1bda831e1d74cf8a74ec8e07c3c560ab.png

Link to comment
On 10/2/2023 at 7:47 PM, itimpi said:

There is a SpaceInvader video about this (do not have link to hand) so that is probably worth looking at as there is no way it should be filling up if all containers are correctly configured to point to mapped volumes for their working storage. 

For those who stumble upon this topic in the future, here's the video.

 

Link to comment
23 hours ago, semioniy said:

Hi. @Hoopster, does this rule of thumb apply to everyone, or only to those who doesn't use UnRaid as a virtualization server with 25 containers running? 😁

image.png.1bda831e1d74cf8a74ec8e07c3c560ab.png

The docker.img file is used to store the executable code for each container.  Even with 25 containers, 20GB is generally enough, if the containers are properly configured; there may be exceptions if some of the containers are unusually large.  At one time I had 20 docker container installed and was using only ~12-13B in docker.img.  A couple of those were 1-2GB in size. 

 

In the Docker tab you can click on the image.png.2cc1f00a02eda930ce405bbf54f46b73.pngbutton to see how much space each container is occupying in docker.img. If some look unusually large, it is likely that data is being stored in the container due to improper configuration

Edited by Hoopster
Link to comment

Well, I ran the script from SpaceInvader1, and it didn't find any problems, and could free respectable 175KB of space 😁
So, I guess I'll assume it's all good. 

BTW,

On 11/22/2023 at 3:56 PM, Hoopster said:

 At one time I had 20 docker container installed and was using only ~12-13B in docker.img.  

doing simple math - 14GB would be 70% of default 20GB, and crossing this threshold would give you a utilization warning. Having 13GB with 20 containers installed, 1 or 2 additional containers (or even a container updating) could already give you a warning, forcing you to either increase the utilization level warning threshold, or enlarge your docker image. 

So, I think I'll chill and won't worry too much about not following the "20 GB should be enough" rule, given how much I love to play around with docker 😁

Edited by semioniy
Link to comment
  • 3 months later...
On 11/22/2023 at 3:56 PM, Hoopster said:

The docker.img file is used to store the executable code for each container.  Even with 25 containers, 20GB is generally enough, if the containers are properly configured; there may be exceptions if some of the containers are unusually large.  At one time I had 20 docker container installed and was using only ~12-13B in docker.img.  A couple of those were 1-2GB in size. 

 

In the Docker tab you can click on the image.png.2cc1f00a02eda930ce405bbf54f46b73.pngbutton to see how much space each container is occupying in docker.img. If some look unusually large, it is likely that data is being stored in the container due to improper configuration

I've got 52 running containers with 107GB in use. It isn't growing except for whilst running updates.
Most containers that use a lot of space, are the ones that already have large Images. How would you honestly stay below the 20GB? Isnt the complete installation of the image also in the docker image? The executables?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.