Understanding Docker Image Utilization

[tkohhh]

I'm having a hard time making sense of the Docker Image Utilization.

 

Label: none  uuid: 18f96c1b-7688-476d-82c1-ab447f3cecbe
	Total devices 1 FS bytes used 29.79GiB
	devid    1 size 40.00GiB used 35.07GiB path /dev/loop2

According to the "Docker volume info" on the Settings>Docker page, I'm using 29.79GiB (or maybe 35.07GiB?) out of 40.00GiB.  The Docker memory display on the Dashboard shows 78%.  This math doesn't work out, but my bigger concern is the "Container Size" window in the Docker tab. 

Name                              Container     Writable          Log
---------------------------------------------------------------------
home-assistant                      1.40 GB      2.65 kB       199 kB
duplicati                            579 MB       303 kB      47.3 kB
PlexMediaServer                      484 MB       337 kB      21.2 kB
mariadb-nextcloud                    366 MB       340 kB      95.8 kB
nextcloud                            332 MB      43.5 kB       141 kB
letsencrypt                          280 MB      13.9 MB       218 kB
Influxdb                             258 MB          0 B       516 kB
Grafana                              233 MB         10 B          0 B
Bitwarden                            104 MB          0 B      57.6 kB
telegraf                            87.6 MB      1.55 MB       115 kB

When I add up the "Container" column and the "Log" column, I get 4.12GB.  Obviously that's nowhere close to 29.79GiB (or 35.07GiB).

 

The reason this came up is I recently deleted a Splunk container that seemed to be eating the image size quite quickly.  Prior to installing the container my image size was set to 20GiB, and my Utilization was around 50%.  I expected that deleting the container would relieve the utilization, but clearly that hasn't happened.  I did make sure that all of my orphaned images are deleted, so that shouldn't be contributing to the utilization.

 

I think I'm not understanding something completely, so I'm hoping somebody can shed some light on what's going on here.

 

Thanks for your help!

Edited May 9, 2020 by tkohhh

[trurl]

20GB docker image should be more than enough. You almost certainly have some application configured so it is writing to a path that isn't mapped. Which dockers do you run?

 

Go to Tools - Diagnostics and attach the complete diagnostics ZIP file to your NEXT post.

[tkohhh]

The only new container was Splunk, and I deleted that prior to this post.  I've done a little bit of tweaking with Home Assistant and LetsEncrypt, but nothing I hadn't done previously without any issues.

 

I run LetsEncrypt, Bitwarden, Duplicati, Grafana, Home Assistant, Influxdb, Nextcloud, Mariadb (for Nextcloud), Plex, and Telegraf.

 

Still, am I missing something with the math of how the Utilization is reported? What makes up the 29.79GiB that "Docker Volume Info" reports?  If the "Container Size" report only shows 4.12G, what is taking up the other 25G?

 

Attached is my diagnostics.  Thank you for your help!

 

server1-diagnostics-20200509-1547.zip

[trurl]

Is Nextcloud storing its files in the mariadb database or somewhere else?

[tkohhh]

In the mariadb database.

[trurl]

Post docker run command for plex as explained at this very first link in the Docker FAQ:

 

https://forums.unraid.net/topic/57181-docker-faq/?do=findComment&comment=564345

 

[tkohhh]

Here it is.

[trurl]

And what do you have for the transcode directory within the plex application? Do you use the plex DVR feature?

[tkohhh]

Here are the settings.  I haven't touched these since I set up Plex about a year ago.

 

And no, I don't know anything about Plex DVR.

 

Edited May 10, 2020 by tkohhh

[tkohhh]

While trurl is troubleshooting my Docker Image Size, I'm curious if anybody else can explain the discrepancy between my Docker Image Utilization and the data reported in "Container Size"?  I'm sure there's something that I'm missing, but my Docker Image is 40GiB, and showing 78% Utilization, which is 31.2GiB.  However, adding up the data in Container Size only equals about 4GiB.  Where does that extra 27.2GiB come from?

[trurl]

I don't use the plexinc docker, I am using Linuxserver.io plex instead. But your plexinc docker mappings don't make sense to me. Why do you have 3 different container paths mapped to the same host path "/mnt/user/appdata/PlexMediaServer"? I just looked at the plexinc documentation on dockerhub, and your mappings don't seem to conform to those docs either.

 

What is the purpose of the container path "/data" in your mappings? And wouldn't it make more sense if the "/transcode" container path was mapped to something other than the exact same path "/config" (your plex library) is mapped to so your transcodes aren't just dumped into that same folder with your plex library?

 

None of that would cause filling docker image problem, but it does make me wonder if you know how to setup dockers.

 

The reason I picked on NextCloud and Plex is because I am using those myself and have some idea what they are and how to set them up. One of the others you are running may be the cause of your problems but it would take a lot more research for me to figure out what you may have done wrong with those.

 

Do you understand docker mappings of host paths to container paths? Do you know Linux is case sensitive? Do you know to use absolute and not relative paths to refer to the container paths within the application? Do you know why I'm asking these questions and what their importance is?

[tkohhh]

First of all, thanks for your efforts on this.  I see that you put a lot of time into helping people out on this forum, and I know that's not always easy.

 

To answer your questions, yes, I do understand the mappings, and I do understand that Linux is case-sensitive.  When not using a template, I have always used absolute paths, because I have no idea what a relative path would be relative to.  And I do understand the importance of these questions and why you're asking them.

 

It's funny that you mention the mappings on the Plex container, because I was looking at that this morning and wondering why it's done that way.  As I mentioned before, the Plex container is one of the first that I ever set up about a year ago, and to be fair I knew a lot less then than I do now.  I checked the first flash backup I have (from May 9, 2019), and the template was the same then.  Who knows why I set it up that way.  What I do know, however, is that Plex has been working well this whole time.

 

I'm an IT professional, so I'm fairly good at troubleshooting in general. Here are the facts of my situation:

 

-I first installed Splunk on Thursday (5/7)

-Prior to installing Splunk, I had my Docker Image set to 20GB, and my reported utilization was less than 60%

-After installing Splunk, I got a warning on my image size, so I increased it to 25GB

-After adding a few logs to Splunk, I was getting warnings again.  I increased the image size to 40GB

-Saturday morning (5/9) I deleted the container after getting frustrated with time zones and also getting a warning that I was over 80% utilization.

-After deleting the container, utilization reduced to 78%

 

So, before Splunk, I was using around 12GB of my docker image, and then after deleting Splunk I'm using around 31GB of my docker image.  That leads me to believe that somehow something from Splunk is sticking around.  Looking at the Container Sizes confirms that my existing container sizes don't add up to anything close to 31GB.

 

So what am I missing?  Am I incorrect in assuming that my Container Sizes should be taking up the majority of my Docker Image utilization?  Is there something else I should be taking into consideration?  Is this just a great mystery that I can only resolve by deleting my docker image and then recreating my containers?

Edited May 10, 2020 by tkohhh

[itimpi]

Rather than trying to solve the utilisation issue, it might be easier and quicker to:

• stop the docker service
• delete the current docker image
• set the size you want for Docker.img
• restart the docker service to create a new empty docker.img
• use the Previous apps option on the Apps tab to reinstall the containers you now want with all their settings intact.

that does not mean that knowing the answer to the original question is not of value For future reference.

[tkohhh]

29 minutes ago, itimpi said:

that does not mean that knowing the answer to the original question is not of value For future reference.

Since the image is staying stable at 78% and everything is functioning correctly, I'm willing to keep troubleshooting this for the sake of the knowledge to be gained.  Since I'm not capable of doing it on my own, however, I'll defer to the community here.  If there is interest in getting to the bottom of this, I'll gladly put in the effort needed on my end to help find the problem.  I just need somebody to keep pointing me in the right direction.

 

Eventually I want to take another stab at Splunk, but I want to have the image size under control before I tackle that.  I'm not in a hurry though, so we've got time to work this out if the community here is willing to help me!

[tkohhh]

Seems like nobody has the appetite to dig into this one, so I'll be deleting my Docker image and then re-adding my templates.

 

Still, I'd like it if somebody could please answer this one question: Should I expect the data reported in "Container Size" on the Docker tab to add up to the Docker utilization reported on the Unraid Dashboard?

[primeval_god]

20 hours ago, tkohhh said:

Still, I'd like it if somebody could please answer this one question: Should I expect the data reported in "Container Size" on the Docker tab to add up to the Docker utilization reported on the Unraid Dashboard?

I am not 100% sure but i believe the answer is NO. The Docker image is where containers (both running and stopped) store their internal file systems, as well as where docker stores Volumes (both the intentionally created and those created by images having volume directives that are not mapped to a bind/mount by the template. Docker Images are also stored there as well as other things used by the docker daemon. 

[tkohhh]

If that's the case it certainly makes sense.  Is there a way to see what that other stuff is?  How can I account for the data between what's reported in my total image size and the "Container Size" report? 

Edited May 12, 2020 by tkohhh

[primeval_god]

Unfortunately I am not certain of the answer to that. Maybe by reading up on docker (via their official documentation). Basically any data that would normally be persisted by docker is stored in the docker image on unRAID.