Docker Image

[tomwhi]

Hi,

 

I'm seeing my unraid server has a high docker memory usage. I have two questions. 

 

I think I've terribly misconfigured my dockers as I've been setting them up because stuff like plex is at 9.3GB and others are higher than I imagined.  I assume it's because I've been careless when choosing where to put the /config directly and allowing it to be put into the appdata folder.  For Plex I can see my metadata folder is massive - I can confirm I don't have any media hosted in there - just the /config directory is massive. 

 

This looks to be the same case for all my other dockers where I've not put any thought into where the config dir should live.

 

Should I move my /config directory out of the appdata or is it ok in there?

 

Is there any issue with increasing the size of the docker.img to accommodate these large folders?

 

 

Cheers

Tom 

 

 

[trurl]

You are confusing 2 separate things, the appdata share, and the docker image.

 

Typically the "config" folder for a docker is mapped to appdata. Plex appdata can become a bit large and what you have seems fine.

 

On the other hand, docker image should not grow and 20G should be more than enough. Making it larger won't help, it will just make it take longer to fill.

 

Go to Tools - Diagnostics, attach complete Diagnostics ZIP file to your NEXT post.

 

Also, docker run command for your plex docker as explained in this very first link in the Docker FAQ:

 

https://forums.unraid.net/topic/57181-docker-faq/?do=findComment&comment=564345

 

Also, the transcode directory setting within the plex application.

[tomwhi]

Thanks for the reply trurl. I think I jumped to some false conclusions about the app data and docker image, thanks for correcting me. 

 

I've attached the diag file from  my server. 

 

This is the run command from the container (ignore the test environment variable) 

 

root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='binhex-plexpass' --net='host' -e TZ="Europe/London" -e HOST_OS="Unraid" -e 'TRANS_DIR'='/config/transcode' -e 'UMASK'='000' -e 'PUID'='99' -e 'PGID'='100' -e 'Test'='Test' -v '/mnt/user/Movies/':'/movies':'rw' -v '/mnt/user/TV/':'/tv':'rw' -v '/mnt/user/Music/':'/music':'rw' -v '/mnt/user/Music-Unsorted/':'/MusicUnsorted':'rw' -v '/mnt/user/Fitness/':'/fitness':'rw' -v '/mnt/user/Home Videos/':'/HomeVideos':'rw' -v '/mnt/user/Training/':'/training':'rw' -v '/mnt/user/Audio Books/':'/audio-books':'rw' -v '/mnt/user/appdata/binhex-plexpass':'/config':'rw' 'binhex/arch-plexpass' 

925f49e97196c5af9b5d66a0f19f21bd19cb36e01f43c6c56b92b713c544cfad
The command finished successfully!

 

 

 

 

tomnas-diagnostics-20200529-1845.zip

[trurl]

That looks OK, depending on your answer to this:

1 hour ago, trurl said:

Also, the transcode directory setting within the plex application.

Plex doesn't write much except trancodes and possibly DVR if you use that. Do you use DVR in Plex?

 

Maybe one of your other dockers is to blame, such as a downloader since they write a lot of data.

 

There are basically 2 ways you can cause a container to write where it shouldn't.

 

If the host path in a mapping for any container isn't a disk or user share, then that host path is in RAM just like the rest of the Unraid OS, and if the application writes to the corresponding container path, it is writing into RAM.

 

More relevant for your issue, though. If an application writes to a path that doesn't correspond exactly to a mapped container path, then those writes go into the docker image. It is important to remember that Linux is case-sensitive and so case matters when specifying a path. And you must also make sure any application paths are absolute (or possibly relative to another absolute path already specified).

 

Specifying the wrong upper/lower case, or specifying a relative path, within the application settings is the most common reason for filling docker image. Note that this isn't really about the mappings, it is about the paths specified within the application itself not matching the mappings.

[tomwhi]

I'm not using DVR in Plex - I have probably set it up at some point but I never really got any use out of it.  My transcode directory doesn't look like it's configured correctly, I thought I have pointed this at a share on my cache drive. 

 

I have had a quick look through all my container settings and it seems they are all setup correctly using the case I set my shares to, i.e. "Download" and not "download" (this is massively helped by the unraid GUI doing the path creation for me.  I've mainly used Binhex's applications so I can't imagine the paths in the application are incorrect, but maybe I'm misunderstanding something. 

 

Trying to find the main culprit I used the cAdvisor (based on another post) to see the virtual size of my dockers and it seems that the ones that reference my "Download" share are the ones with higher usage.  I don't know if this is the right way to go. 

 

 

I drilled into Sonarr (just to pick one) and the directories look correct 

 

 

The settings in docker seem to match my shares... 

 

 

 

Are there any ways to look inside the docker.img file so I can see what the kind of files are that are in there, so then I can reverse engineer the problem and try to work out what in the applications are misconfigured?

[trurl]

5 hours ago, tomwhi said:

The settings in docker seem to match my shares... 

Not sure why you are telling me that. Perhaps you don't understand my explanation above. It is the settings within the application itself that must exactly match your mappings, specifically the container path in your mappings.

 

For example, if your deluge docker has /download mapped to /mnt/user/Download, but within the deluge application itself you have it set to use /Download, then that won't work because /download and /Download are different paths. The application must use the path specified as the container path in the mapping.

 

[tomwhi]

Yes - sorry, I have read your post again and see what you mean.  I have been through SabNZB, Deluge, and others and they look ok to me.  The cases match what they should be (I use the auto complete where possible so I don't think there is a problem with case).  I can see why this would be normally the case but this seems to have been creeping up slowly for me so I'm not 100% sure it's the case. 

 

When I total my docker sizes vs what unraid is reporting then the two totals don't add up so I think there's more in the docker image than just the live dockers I have installed. 

 

To dig around some more, I looked inside /var/lib/docker and with the command "du -ah /var/lib/docker/containers/ | sort -rh | head -60"  I could see containers in there which I no longer have; is there a way to safely remove these and just leave the dockers I am actually running?

 

I have a bad habit of adding and removing dockers quite a lot so I wonder if the removal process isn't removing the old images?

 

 

Edited May 31, 2020 by tomwhi