How to find out what's filling up docker image?


Recommended Posts

Ah, gotta love the quick easy answers!  Thanks.

 

Now to take it a step further.  I'm noticing that a lot of my dockers are taking up quite a bit of space in virtual size.  For instance, dockers like Radarr, Sonarr, and Jackett are around 500MB each.  What causes them to be so high?  Is it simply log files?

 

I have abotu 20 dockers, and 30GB that's now full.   All of my dockers come from Community Apps, and I map all the paths according to the templates outside of the containers.

 

I can increase without issue, but I want to make sure I'm not doing something fundamentally wrong.

 

Edited by Living Legend
Link to comment

Docker containers should not contain any data, they are supposed to be static images and all data should be on externally mapped paths. At least that's my understanding. So if container size is growing something is wrong. Even with log files they should be getting pruned periodically.

  • Upvote 2
Link to comment

Hmm, off topic, but for some reason I can't get the docker started again.  I stopped it, tried to change size from 30-40, and then started again.  It wouldn't start so I reverted back to 30, but now I can't get the docker service to run.  The image file still exists in the location it's pointed to, but it's not starting.

 

EDIT: Apparently I'm a liar.  I stopped it again, expanded to 35, started, and now it's running.

Edited by Living Legend
Link to comment

80%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img.

15%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img due to a capitalization error on a folder (ie: /downloads is mapped, but container references /Downloads)

5% of the time its due to excessive logging by a container.

 

Virtual Sizes AFAIK are only the sizes of the actual container, and nothing else that the container might create within the image.  Net result is that its a pretty useless metric, but since the docker system has it available, I make it visible.

 

You can peruse the Temporary Docker FAQ for some suggestions, or the investigation regarding out of control apps logging "Hi! I'm still running.  How are you doing?" every second here:http://lime-technology.com/forum/index.php?topic=45249.0

 

 

  • Upvote 2
Link to comment

 

Exactly what I followed and I've used it on every Docker so far and Haven't noticed my Image grow at all.  Just make sure you enable Advanced Settings and plug it into the Text Box. 

 

--log-opt max-size=50m --log-opt max-file=1
 

Specifying the max size without the max-file won't actually do anything because all its going to do is archive the old logs and recreate another.  Specifying max-file will delete any old logs.

 

Link to comment
22 hours ago, kizer said:

 

Exactly what I followed and I've used it on every Docker so far and Haven't noticed my Image grow at all.  Just make sure you enable Advanced Settings and plug it into the Text Box. 

 


--log-opt max-size=50m --log-opt max-file=1
 

Specifying the max size without the max-file won't actually do anything because all its going to do is archive the old logs and recreate another.  Specifying max-file will delete any old logs.

 

 

By doing this I went from essentialy full, to 10GB free.  Thanks!

 

Out of curiosity, I'm running about 20 dockers, and right now around 25GB in space.  Does this seem appropriate, or does it smell of some improper mapping somewhere?

Link to comment

I personally assigned 25GB to my image and I was very careful with my mapping to make sure everything pointed to:

folder for data - /mnt/cache/docker-name-here  now that I think of it I should of created a folder /mnt/cache/dockerdata/docker-name-here lol 

for configuration files - /mnt/cache/system/appdata/docker-name-here        

 

I think I have roughly 10 dockers installed, but I only have 5 that are currently running/active.

Link to comment
5 hours ago, kizer said:

I personally assigned 25GB to my image and I was very careful with my mapping to make sure everything pointed to:

folder for data - /mnt/cache/docker-name-here  now that I think of it I should of created a folder /mnt/cache/dockerdata/docker-name-here lol 

for configuration files - /mnt/cache/system/appdata/docker-name-here        

 

I think I have roughly 10 dockers installed, but I only have 5 that are currently running/active.

Assigned 25GB.  What are you using of the 25?

Link to comment

I definitely have something incorrectly configured because I've gone from 10GB free to 3GB free without adding any dockers.

 

If I were to delete dockers and image one by one and pay attention to available space in the allocated docker image, would that be a way to tell which one is hogging up all the space?

Link to comment
21 minutes ago, Living Legend said:

I definitely have something incorrectly configured because I've gone from 10GB free to 3GB free without adding any dockers.

 

If I were to delete dockers and image one by one and pay attention to available space in the allocated docker image, would that be a way to tell which one is hogging up all the space?

My gut is telling me no, but it can't hurt to try. Have you noticed any patterns in the usage increase from what items are being downloaded?

Link to comment
18 minutes ago, jonathanm said:

My gut is telling me no, but it can't hurt to try. Have you noticed any patterns in the usage increase from what items are being downloaded?

 

I actually tried.  Odd results.  I would delete a docker and my total available space would decrease until I re added it.  No idea there.

 

And as for patterns, I've been trying to find some, but having issues.  I'm having another issue that I thought was unrelated, but maybe not?  All of my downloads go to the /mnt/user/downloads share which uses the cache drive.  That share is set to use the Cache disk.  They are then renamed/moved by their respective docker into /mnt/user/movies or /mnt/user/tv which are both in the array, and set to no for cache.  But for some reason, movies are sitting in /mnt/cache/movies or rather than /mnt/user/movies after the mover runs.

 

Likely unrelated, but who knows.  And now with Radarr, I have an issue because the available space is so low that when it creates a backup of a movie when renaming/movie it runs out of space.

 

Couldn't import movie /unraid/downloads/movies/MOVIENAME: Disk full. Path 

One other thought is zoneminder, but all of the data is showing up in /mnt/user/surveillance so I'm not sure what would be remaining inside the image.

Link to comment
2 hours ago, Living Legend said:

And now with Radarr, I have an issue because the available space is so low that when it creates a backup of a movie when renaming/movie it runs out of space.

AHA! Sounds like your issue is with Radarr. It shouldn't be using space inside the docker to rename / move.

 

Check all the paths referenced in Radarr, make sure they ALL properly point to mapped locations.

Link to comment
44 minutes ago, Living Legend said:

Notice anything?

 

Screenshot_1.thumb.png.e1b9069b0ae177bd8eebe2e97b614eb1.png

I don't use this docker, but what you need to check is that nothing in the application itself is trying to save to a location other than those you have mapped; i.e., /downloads, /movies, /unraid, /config. And of course the case matters, as does the /

 

Link to comment
11 minutes ago, trurl said:

I don't use this docker, but what you need to check is that nothing in the application itself is trying to save to a location other than those you have mapped; i.e., /downloads, /movies, /unraid, /config. And of course the case matters, as does the /

 

 

Hard to say.  I tend to stick to the templates.  For something like Radarr, the only thing it should be downloading are .nzb and .torrent files.  Then the downloader picks up those and downloads the large media files.

 

/downloads is mapped so it knows where to find the downloads so it can rename and move

/movies is mapped so it knows my current media library, and can rename and move the file from /downloads (cache) to /movies (array)

/config is the standard appdata folder

/unraid is there to give it access to anything on the array

 

Not sure what else it could besides what @jonathanm mentioned about the backup file it's creating.  Not sure where it's placing that, and if it's deleting it.  Nothing in the template mentions it so I don't know.

 

I don't think it's that though because I haven't downloaded any movies today and I'm down 1GB in size.  That makes me think it's either Zoneminder for the cameras, or possibly Plex DVR recording a couple TV shows?

 

EDIT:  Maybe Sonarr?  Just realized there's this /dev/rtc mapped to something that I'm not sure what it even is....

 

 

mappings.png

Edited by Living Legend
Link to comment
15 minutes ago, Living Legend said:

Hard to say.  I tend to stick to the templates.  For something like Radarr, the only thing it should be downloading are .nzb and .torrent files.  Then the downloader picks up those and downloads the large media files.

What everyone is asking you for is to check the settings within Radarr's UI (or post screenshots).

 

16 minutes ago, Living Legend said:

EDIT:  Maybe Sonarr?  Just realized there's this /dev/rtc mapped to something that I'm not sure what it even is....

 

Don't worry about it - its passing the date&time

Link to comment
14 minutes ago, Squid said:

What everyone is asking you for is to check the settings within Radarr's UI (or post screenshots).

 

Don't worry about it - its passing the date&time

 

Within the UI the only path I can see that I've set is drone factory:

 

mappings2.thumb.png.b5c5430fd7c5e61d6e75ed0ec9e33d01.png

 

Usually to be safe I don't type in the path, but rather click the folders and work my way down to make sure that I'm using the proper format.

Link to comment
  • 2 years later...

came across this thread for docker utilization: "what's taking up all my docker.img space in there?"

 

"Resource Monitor" in Community Applications has been replaced by cAdvisor, which is in the APPS tab. Go to SERVER:8085, click Docker Containers, at the top, scroll to bottom, and sort by Virtual Size. You can see in the image below, Krusader is hogging up 2.29GB

GpJMi02m.png

 

 

 

  • Thanks 1
Link to comment
  • 1 year later...
On 6/7/2017 at 4:09 PM, Squid said:

80%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img.

15%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img due to a capitalization error on a folder (ie: /downloads is mapped, but container references /Downloads)

5% of the time its due to excessive logging by a container.

 

Virtual Sizes AFAIK are only the sizes of the actual container, and nothing else that the container might create within the image.  Net result is that its a pretty useless metric, but since the docker system has it available, I make it visible.

 

You can peruse the Temporary Docker FAQ for some suggestions, or the investigation regarding out of control apps logging "Hi! I'm still running.  How are you doing?" every second here:http://lime-technology.com/forum/index.php?topic=45249.0

 

 

so I have confirmed my docker mapping, and I have log sizes limited and have confirmed they are not the issue. I'm pretty sure I'm in the 80% block. but i have a 100gb docker image file and I'm at 71% full now. 

 

Any advice on how I can root out what is causing this issue?

Link to comment
9 minutes ago, Aerodb said:

does this app still exist? what name has it changed to?

No.

 

You can get the container sizes right from the docker tab (Container Size button) to see which app is taking up which space (not quite 100% accurate, but will give a good idea)

  • Like 1
  • Thanks 3
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.