assassinmunky Posted September 13, 2017 Share Posted September 13, 2017 On 9/7/2017 at 8:06 AM, syrys said: For those that know about docker, is there really no way for one to browse into a docker containers file system and inspect/delete a file? Say you want to locate and delete a log fine manually, and you are sure its just being saved inside the docker, is there really no way to browse into the docker's file system and try locate and remove said file(s)? This would solve most of our problems (although hackey, its still a viable solution). Sure there is, you can "enter" the docker container with this command: docker exec -i -t CONTAINER_ID /bin/bash Get the CONTAINER_ID from the "docker ps" command Quote Link to comment
BRiT Posted September 13, 2017 Share Posted September 13, 2017 You do NOT need the container_id at all. you can use the Container Name. Quote Link to comment
Rivvern Posted March 5, 2018 Share Posted March 5, 2018 Ok. Been having this problem for a long time but never actually looked into problem. I just solved it by increasing the image size. But with a docker image being almost 60gb its time to fix the problem. Im Windows guy, having little knowledge in linux commands so i need expert help. I have the cadvisor container but i cant find anything useful in it. Somewhere i read that have the mapping /mnt/user is a bad thing...? Can you see what my problem is? I attached an image of the configuration. Please let me know if you need anything else. Quote Link to comment
trurl Posted March 5, 2018 Share Posted March 5, 2018 1 hour ago, Rivvern said: I attached an image of the configuration. As already mentioned in this thread and the Docker FAQ, in addition to having the container mappings correct, you must check that each application is only writing into folders that are mapped; i.e., on the left side of those mappings you posted. Here is the Docker FAQ, see the section about Docker Image Filling Up https://lime-technology.com/forums/topic/57181-real-docker-faq/ 1 1 Quote Link to comment
Mikki Posted March 6, 2018 Share Posted March 6, 2018 (edited) I had a problem with sync, it filled the disk with a log file (80gb+) i was located in /var/lib/docker/containers/Some"Random"FolderNameofHEX/SameNameAsBefore.json try look for that Edited March 6, 2018 by Mikki 2 Quote Link to comment
Rivvern Posted March 6, 2018 Share Posted March 6, 2018 (edited) Ok so i found out that i had ALOT of orphaned images. I read this and then i removed them all with this help.. https://forums.docker.com/t/how-do-i-remove-orphaned-images/1172 Why do you think i had so many orphaned images? Edited March 6, 2018 by Rivvern Quote Link to comment
Squid Posted March 6, 2018 Share Posted March 6, 2018 5 hours ago, Rivvern said: Why do you think i had so many orphaned images? Orphaned images only tend to happen when a container fails to install and run (ie: conflicting port, etc). 1 Quote Link to comment
Rivvern Posted March 14, 2018 Share Posted March 14, 2018 Ok so it seems like i keep getting orphaned images. Noticed another thread where some guy got the same problem but there are no solution. https://lime-technology.com/forums/topic/65756-orphan-images/ I cannot find any issues with my container settings. If anyone know how to fix it or how to troubleshoot id be grateful. Quote Link to comment
Squid Posted March 14, 2018 Share Posted March 14, 2018 26 minutes ago, Rivvern said: Ok so it seems like i keep getting orphaned images. Uninstall advanced buttons plugin 1 Quote Link to comment
Rivvern Posted March 14, 2018 Share Posted March 14, 2018 1 hour ago, Squid said: Uninstall advanced buttons plugin Thanks, but ive never had that one installed afaik. Quote Link to comment
Energen Posted March 18, 2018 Share Posted March 18, 2018 I've recently been getting warnings also... and it would appear that, at least in my case, btrfs is the culprit. root@UNRAID:~# du -h -d 1 /var/lib/docker/ 16M /var/lib/docker/containers85G /var/lib/docker/btrfs 26M /var/lib/docker/image 46M /var/lib/docker/volumes 0 /var/lib/docker/trust 88K /var/lib/docker/network 0 /var/lib/docker/swarm 2.4M /var/lib/docker/unraid 0 /var/lib/docker/tmp 0 /var/lib/docker/plugins 16K /var/lib/docker/builder 85G /var/lib/docker/ 85GB for....... what exactly? I've seen a number of sites talking about this issue and it seems that most people end up rebuilding their dockers in order to reduce the size....... I'm not interested in doing that, at all. https://github.com/moby/moby/issues/27653 https://gist.github.com/hopeseekr/cd2058e71d01deca5bae9f4e5a555440 https://github.com/moby/moby/issues/9939 Short of simply increasing the size of the docker as @Squid has outlined, is there any way to non-destructively "refresh" btrfs to reduce it's size? Quote Link to comment
Squid Posted March 18, 2018 Share Posted March 18, 2018 Most causes of docker.img getting that big is either out of control logging by the apps downloading directly into the image Both circumstances are covered in the docker FAQ 2 Quote Link to comment
Energen Posted March 18, 2018 Share Posted March 18, 2018 (edited) 1 hour ago, Squid said: Most causes of docker.img getting that big is either out of control logging by the apps downloading directly into the image Both circumstances are covered in the docker FAQ As far as I can tell neither of those situations apply to me. Any docker app that has a directory to configure is mapped to a share. I don't think there's anything downloading to the image. Logs... du -ah /var/lib/docker/containers/ | grep -v "/$" | sort -rh | head -60 Returns about 20MB of logs. Not sure what else to look at? Edit -- I examined /var/lib/docker/btrfs/subvolumes and there are 167 directories, far more than apps/dockers I've got installed if that matters... some which are quite large. Example: 1.2G 60ba2059df9b6ee30f3dac1084f7041bc5ff0018c77d34cef3d71aede6ba7076 1.2G fd00c8da84099fb7c380e06ff82feccdc87862706ef8b371a86c5afaf7e66617 985M bcd719fbcb62e15f4bb96fe56b441d005a55dede653fd49cfe9b4400f4e482f8 985M c6658bfc2643ad274c31a16229f8bb77edf257ccbd712635cd87c7f325752163 So how do I go about figuring out what is what and cleaning some of it up? I can't possibly need all of this to be here.... 167? I've got 14 dockers and 4 VMs.. seems excessive. Edited March 18, 2018 by Energen Quote Link to comment
bonienl Posted March 18, 2018 Share Posted March 18, 2018 2 hours ago, Energen said: 85G /var/lib/docker/btrfs This is the size of your Docker image file (80G). How much percentage is used as reported on the Dashboard? Quote Link to comment
Energen Posted March 19, 2018 Share Posted March 19, 2018 12 hours ago, bonienl said: This is the size of your Docker image file (80G). How much percentage is used as reported on the Dashboard? It's currently 37%, after I deleted 2 dockers (Krusader and Dolphin) which also brought my btrfs subvolmes size down to 54GB. The warnings I had been getting were docker image utilization over 70%, as high as 85% at one point. Quote Link to comment
trurl Posted March 19, 2018 Share Posted March 19, 2018 Not sure what but you definitely have something not working normally. Most people never get that warning with only 20GB docker image. Quote Link to comment
JonathanM Posted March 20, 2018 Share Posted March 20, 2018 On 3/18/2018 at 5:41 PM, Energen said: Any docker app that has a directory to configure is mapped to a share. That's only partially relevant. What can happen is this. Say you have /data mapped to /mnt/user/downloads. So far, so good. But in the app itself, instead of working totally in /data, the app has a folder pointing to /downloads or something. Anything NOT saved to the mapped folder INSIDE the docker app will download into the docker image itself. 1 Quote Link to comment
bonienl Posted March 20, 2018 Share Posted March 20, 2018 On 3/19/2018 at 11:50 AM, Energen said: It's currently 37%, after I deleted 2 dockers (Krusader and Dolphin) which also brought my btrfs subvolmes size down to 54GB. Freeing up so much space when deleting the containers means almost certainly that data was stored inside the container and not into a mapped folder. Like @jonathanm says, you need to carefully check the applications settings itself to make sure it is wrtining to the correct (mapped) destination. 1 Quote Link to comment
Nyghthawk Posted April 2, 2018 Share Posted April 2, 2018 (edited) I went from 79%, found this thread, Did a few commands to see the sizes of my container folders refreshed, 83%, Refreshed 89%. Wow. Where do I start? du -h -d 1 /var/lib/docker/ 7.4G /var/lib/docker/containers 69G /var/lib/docker/btrfs 35M /var/lib/docker/image 52K /var/lib/docker/volumes 0 /var/lib/docker/trust 108K /var/lib/docker/network 0 /var/lib/docker/swarm 2.5M /var/lib/docker/unraid 0 /var/lib/docker/plugins 16K /var/lib/docker/builder 0 /var/lib/docker/tmp 76G /var/lib/docker/ Edited April 2, 2018 by Nyghthawk Quote Link to comment
Squid Posted April 2, 2018 Share Posted April 2, 2018 40 minutes ago, Nyghthawk said: refreshed, 83%, Refreshed 89%. Assuming the refreshes were in fairly quick succession. In that case, either you're downloading to the docker.img, or an app has out of control logging enabled (usually referred to within the app as "debug". Couple of entries in the docker FAQ about all this. Quote Link to comment
Nyghthawk Posted April 2, 2018 Share Posted April 2, 2018 I willl start there. ty Quote Link to comment
vizi95 Posted April 11, 2018 Share Posted April 11, 2018 I had the same issue. It has to do with docker log file sizes. Just run this and your utilization will come down a lot: truncate -s 0 /var/lib/docker/containers/*/*-json.log It's safe to use while the docker is still running. 2 Quote Link to comment
Blaze9 Posted May 8, 2018 Share Posted May 8, 2018 (edited) On 4/11/2018 at 10:23 AM, vizi95 said: I had the same issue. It has to do with docker log file sizes. Just run this and your utilization will come down a lot: truncate -s 0 /var/lib/docker/containers/*/*-json.log It's safe to use while the docker is still running. This will truncate the docker logs, but just to be certain, data of my individual docker containers aren't affected? I'm not sure if any container logs actual data to the -json.log files. Edited May 8, 2018 by Blaze9 1 Quote Link to comment
itimpi Posted May 19, 2018 Share Posted May 19, 2018 The latest unRAID 6.5.2 release has an option to open a command line shell within a running docker container. This might allow you to poke around inside each docker to try and work out what is using up the space. Quote Link to comment
CJW Posted July 12, 2018 Share Posted July 12, 2018 I just started getting warnings about high docker.img utilization and found this thread. The problem is that I have no idea which container might be the culprit. My docker.img is 24G. I have 9 containers installed and "docker ps -s" shows usage of no more than 50MB (5GB virtual). I have bashed into each running docker and run "du -h -d 1 -x /" and I am getting sizes that all agree with the virtual usage from the "docker ps -s" command. I am getting warnings for about 75% usage, which would be about 16GB, so I have no idea where the other 10+GB of usage is coming from. I have also run Container Cleanup and have, to the best of my knowledge, no orphaned containers. I would be happy to fix whatever container is causing the problems but I have no ideas at this point how to figure out where that problem is. Can the docker.img file be mounted somehow to look into it and see what the actual usage is? Help, please! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.