Jump to content
mgranger

Docker Image Filling Up in 6.8.0-rc5

9 posts in this topic Last Reply

Recommended Posts

I don't know if this has to do with 6.8.0-rc5 but my Docker Image seems to be filling up and deleting large files from the appdata folder didn't seem to have any impact.  I am sure I am doing something wrong but I am up to 80% full and I have gone through my applications and made sure the ones that were taking up the most space due to .db files were located on the array.  My docker image is 40 GB and my appdata folder size is only 10.1 GB.  I will post my diagnostics in hopes that this will help.

 

 

finalizer-diagnostics-20191108-1505.zip

Share this post


Link to post
2 hours ago, mgranger said:

I don't know if this has to do with 6.8.0-rc5 but my Docker Image seems to be filling up and deleting large files from the appdata folder didn't seem to have any impact.  I am sure I am doing something wrong but I am up to 80% full and I have gone through my applications and made sure the ones that were taking up the most space due to .db files were located on the array.  My docker image is 40 GB and my appdata folder size is only 10.1 GB.  I will post my diagnostics in hopes that this will help.

 

 

finalizer-diagnostics-20191108-1505.zip 204.62 kB · 0 downloads

Database files in appdata folders have no impact on the size of the docker.img file. Deleting them will not reduce the size of the docker.img.  You undoubtedly have one or more docker containers misconfigured in such way that data that should be stored in folders on the array, an unassigned disk, etc. is instead being stored in the docker image file.  Check all you docker volume mappings and the actual application settings.  

 

Common culprits are file downloaders and backup programs.

Edited by Hoopster

Share this post


Link to post
17 minutes ago, Hoopster said:

Database files in appdata folders have no impact on the size of the docker.img file. Deleting them will not reduce the size of the docker.img.  You undoubtedly have one or more docker containers misconfigured in such way that data that should be stored in folders on the array, an unassigned disk, etc. is instead being stored in the docker image file.  Check all you docker volume mappings and the actual application settings.  

 

Common culprits are file downloaders.

I feel like I have done this.  I have my Windows PC setup to scan the appdata folder to see which applications were taking up the most space.  I found out that it was my Home Assistant and Resilio-Sync which were mapped /mnt/user/appdata/..... so I changed it to /mnt/user/Media/..... which is on the array.  They were both over 6GB each and changing these had no impact on the docker image percent.  Obviously I am still missing something.

Share this post


Link to post
1 hour ago, mgranger said:

I feel like I have done this.  I have my Windows PC setup to scan the appdata folder to see which applications were taking up the most space.  I found out that it was my Home Assistant and Resilio-Sync which were mapped /mnt/user/appdata/..... so I changed it to /mnt/user/Media/.

Yes, you are still missing something 😁

 

Don't worry, this is very common in the beginning for those trying to wrap their heads around how docker volume mappings work.

 

The size of individual folders in appdata has NO BEARING on the size of the docker.img file which is just a file that happens to be stored (unless you changed it) in the appdata folder.

 

When I referred to mis-configured docker containers I was talking about the internal storage mappings in the container itself and in the application (Home Assistant and Resilio Sync in your case).  This has nothing to do with the appdata path for storing the application config files.

 

I have 19 docker containers configured.  All of them are mapped to /mnt/user/appdata{name of container} as the path for the application configuration files.  These are configuration files and databases only and not the actual application.  That is in docker.img.

 

With 19 docker containers, I have a 20GB docker.img file and less than 13GB is currently occupied and does not grow unless I add new containers.

 

I don't use Home Assistant or Resilio Sync so I can't comment on the volume mappings (found in the container Edit screen) they may require.

 

I'll give you an example with DelugeVPN which downloads content.

 

Here is my volume mapping in the DelugeVPN Edit screen for the data storage path:

image.thumb.png.8bc62453678cb0003ed00de65adb4357.png

 

I mapped the specific location on my unRAID server (the /mnt.../DelugeData folder) to /data in the container.  It is VERY important that you remember what the container path is when setting up the application.

 

Deluge downloads are set to go to a DelugeData folder which I have specified as a sub folder of appdata.  It does not have to go there, I decided that organizationally, that's where I wanted it.  However this is only half of the equation.  This path will NEVER be used unless I tell the application to use it.

 

In the Deluge application Preferences/Downloads configuration, I have specified downloads in progress to go to /data/incomplete.  There is NO /data folder on my server, but in the mapping above, I told Deluge to see /data as the host path /mnt/cache/appdata/DelugeData.  Because of this, Deluge now know that downloads in progress should be stored in  /mnt/cache/appdata/DelugeData/incomplete.  When the download is complete it then gets moved to /mnt/cache/appdata/DelugeData/completed.

 

image.png.414cf5bdcce7dd4402754af1ee30baff.png

 

If I do not tell the application to use the volume mapping I set up in the docker container edit screen, it would just go to whatever it defaults to which, more than likely, is going to result in an application path that end ups inside of docker.img because that is where the actual application lives.  This is why your docker.img file is growing.  Your applications are storing data in docker.img instead of a designated file location on the server.

Share this post


Link to post

Ok I think that makes sense to me but I am still wrapping my head around it a little.  So is there any easy way to check what application is mapped incorrectly?  I have deluge as well but rarely use it and I think it is mapped correctly.  Just having trouble figuring out where the issue is exactly.

Share this post


Link to post
10 minutes ago, mgranger said:

So is there any easy way to check what application is mapped incorrectly?

Click the Container Size button at the bottom of the Docker screen.  It shows you how much space is occupied inside docker.img by each docker container.

 

Look for containers that are unusually large.

 

image.png.5abb1090c20a40db234a7a4f86850dcb.png

Share this post


Link to post

Ok from that I think I found the issue.  The Container size of my Zoneminder is 9.06 GB.  I don't think it is growing anymore as it is routed to a unassigned device but there might be some stuff left over from awhile ago.  What is the best way to get rid of of this stuff without messing up the application settings.  It took me awhile to get everything dialed in and would hate to lose all my settings.  

 

I know these are probably pretty simple answers but it is helping me out quite a bit.

Share this post


Link to post
4 minutes ago, mgranger said:

What is the best way to get rid of of this stuff without messing up the application settings

1.  Disable Docker

2. Delete docker.img and recreate it with a 20GB size

3. Enable Docker

 

image.thumb.png.d54ec58957a349fd03dcfc4e21af2402.png

 

4. Go to the Apps tab (Community Applications) and select Previous Apps from the menu in the upper left.

image.png.a352c725c2623e342a554cbb7a470aa4.png

 

5. Your previously installed docker containers will appear there and you can reinstall them

Note: they will retain all the settings with which they were configured at the time you deleted docker.img

Share this post


Link to post
On 11/8/2019 at 2:06 PM, Hoopster said:

1.  Disable Docker

2. Delete docker.img and recreate it with a 20GB size

3. Enable Docker

 

image.thumb.png.d54ec58957a349fd03dcfc4e21af2402.png

 

4. Go to the Apps tab (Community Applications) and select Previous Apps from the menu in the upper left.

image.png.a352c725c2623e342a554cbb7a470aa4.png

 

5. Your previously installed docker containers will appear there and you can reinstall them

Note: they will retain all the settings with which they were configured at the time you deleted docker.img

This worked perfectly.  Thanks so much for the help

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.