Living Legend Posted June 6, 2017 Share Posted June 6, 2017 Is there an easy way to figure this out? My image is 30GB but for some reason it keeps filling up. I thought I had everything properly mapped so that data would be stored on the cache drive and then moved to the array. But apparently I'm missing something as all 30GB are currently full. Quote Link to comment
trurl Posted June 6, 2017 Share Posted June 6, 2017 Resource Monitor in Community Applications. Quote Link to comment
Living Legend Posted June 6, 2017 Author Share Posted June 6, 2017 (edited) Ah, gotta love the quick easy answers! Thanks. Now to take it a step further. I'm noticing that a lot of my dockers are taking up quite a bit of space in virtual size. For instance, dockers like Radarr, Sonarr, and Jackett are around 500MB each. What causes them to be so high? Is it simply log files? I have abotu 20 dockers, and 30GB that's now full. All of my dockers come from Community Apps, and I map all the paths according to the templates outside of the containers. I can increase without issue, but I want to make sure I'm not doing something fundamentally wrong. Edited June 6, 2017 by Living Legend Quote Link to comment
MrCrispy Posted June 6, 2017 Share Posted June 6, 2017 Docker containers should not contain any data, they are supposed to be static images and all data should be on externally mapped paths. At least that's my understanding. So if container size is growing something is wrong. Even with log files they should be getting pruned periodically. 2 Quote Link to comment
Living Legend Posted June 6, 2017 Author Share Posted June 6, 2017 (edited) Hmm, off topic, but for some reason I can't get the docker started again. I stopped it, tried to change size from 30-40, and then started again. It wouldn't start so I reverted back to 30, but now I can't get the docker service to run. The image file still exists in the location it's pointed to, but it's not starting. EDIT: Apparently I'm a liar. I stopped it again, expanded to 35, started, and now it's running. Edited June 7, 2017 by Living Legend Quote Link to comment
Living Legend Posted June 7, 2017 Author Share Posted June 7, 2017 I took the resource monitor and threw it into excel. The sum of the virtual sizes was under 10GB. So what else is filling up the 30 GB virtual disk I have set for my docker IMG? I'm currently running about 20 dockers. 30 GB filled up so I had to change it to 35. Quote Link to comment
Squid Posted June 7, 2017 Share Posted June 7, 2017 80%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img. 15%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img due to a capitalization error on a folder (ie: /downloads is mapped, but container references /Downloads) 5% of the time its due to excessive logging by a container. Virtual Sizes AFAIK are only the sizes of the actual container, and nothing else that the container might create within the image. Net result is that its a pretty useless metric, but since the docker system has it available, I make it visible. You can peruse the Temporary Docker FAQ for some suggestions, or the investigation regarding out of control apps logging "Hi! I'm still running. How are you doing?" every second here:http://lime-technology.com/forum/index.php?topic=45249.0 2 Quote Link to comment
kizer Posted June 7, 2017 Share Posted June 7, 2017 Exactly what I followed and I've used it on every Docker so far and Haven't noticed my Image grow at all. Just make sure you enable Advanced Settings and plug it into the Text Box. --log-opt max-size=50m --log-opt max-file=1 Specifying the max size without the max-file won't actually do anything because all its going to do is archive the old logs and recreate another. Specifying max-file will delete any old logs. Quote Link to comment
Living Legend Posted June 8, 2017 Author Share Posted June 8, 2017 22 hours ago, kizer said: Exactly what I followed and I've used it on every Docker so far and Haven't noticed my Image grow at all. Just make sure you enable Advanced Settings and plug it into the Text Box. --log-opt max-size=50m --log-opt max-file=1 Specifying the max size without the max-file won't actually do anything because all its going to do is archive the old logs and recreate another. Specifying max-file will delete any old logs. By doing this I went from essentialy full, to 10GB free. Thanks! Out of curiosity, I'm running about 20 dockers, and right now around 25GB in space. Does this seem appropriate, or does it smell of some improper mapping somewhere? Quote Link to comment
kizer Posted June 8, 2017 Share Posted June 8, 2017 I personally assigned 25GB to my image and I was very careful with my mapping to make sure everything pointed to: folder for data - /mnt/cache/docker-name-here now that I think of it I should of created a folder /mnt/cache/dockerdata/docker-name-here lol for configuration files - /mnt/cache/system/appdata/docker-name-here I think I have roughly 10 dockers installed, but I only have 5 that are currently running/active. Quote Link to comment
Living Legend Posted June 9, 2017 Author Share Posted June 9, 2017 5 hours ago, kizer said: I personally assigned 25GB to my image and I was very careful with my mapping to make sure everything pointed to: folder for data - /mnt/cache/docker-name-here now that I think of it I should of created a folder /mnt/cache/dockerdata/docker-name-here lol for configuration files - /mnt/cache/system/appdata/docker-name-here I think I have roughly 10 dockers installed, but I only have 5 that are currently running/active. Assigned 25GB. What are you using of the 25? Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 I definitely have something incorrectly configured because I've gone from 10GB free to 3GB free without adding any dockers. If I were to delete dockers and image one by one and pay attention to available space in the allocated docker image, would that be a way to tell which one is hogging up all the space? Quote Link to comment
JonathanM Posted June 12, 2017 Share Posted June 12, 2017 21 minutes ago, Living Legend said: I definitely have something incorrectly configured because I've gone from 10GB free to 3GB free without adding any dockers. If I were to delete dockers and image one by one and pay attention to available space in the allocated docker image, would that be a way to tell which one is hogging up all the space? My gut is telling me no, but it can't hurt to try. Have you noticed any patterns in the usage increase from what items are being downloaded? Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 18 minutes ago, jonathanm said: My gut is telling me no, but it can't hurt to try. Have you noticed any patterns in the usage increase from what items are being downloaded? I actually tried. Odd results. I would delete a docker and my total available space would decrease until I re added it. No idea there. And as for patterns, I've been trying to find some, but having issues. I'm having another issue that I thought was unrelated, but maybe not? All of my downloads go to the /mnt/user/downloads share which uses the cache drive. That share is set to use the Cache disk. They are then renamed/moved by their respective docker into /mnt/user/movies or /mnt/user/tv which are both in the array, and set to no for cache. But for some reason, movies are sitting in /mnt/cache/movies or rather than /mnt/user/movies after the mover runs. Likely unrelated, but who knows. And now with Radarr, I have an issue because the available space is so low that when it creates a backup of a movie when renaming/movie it runs out of space. Couldn't import movie /unraid/downloads/movies/MOVIENAME: Disk full. Path One other thought is zoneminder, but all of the data is showing up in /mnt/user/surveillance so I'm not sure what would be remaining inside the image. Quote Link to comment
JonathanM Posted June 12, 2017 Share Posted June 12, 2017 2 hours ago, Living Legend said: And now with Radarr, I have an issue because the available space is so low that when it creates a backup of a movie when renaming/movie it runs out of space. AHA! Sounds like your issue is with Radarr. It shouldn't be using space inside the docker to rename / move. Check all the paths referenced in Radarr, make sure they ALL properly point to mapped locations. Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 Notice anything? Quote Link to comment
trurl Posted June 12, 2017 Share Posted June 12, 2017 44 minutes ago, Living Legend said: Notice anything? I don't use this docker, but what you need to check is that nothing in the application itself is trying to save to a location other than those you have mapped; i.e., /downloads, /movies, /unraid, /config. And of course the case matters, as does the / Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 (edited) 11 minutes ago, trurl said: I don't use this docker, but what you need to check is that nothing in the application itself is trying to save to a location other than those you have mapped; i.e., /downloads, /movies, /unraid, /config. And of course the case matters, as does the / Hard to say. I tend to stick to the templates. For something like Radarr, the only thing it should be downloading are .nzb and .torrent files. Then the downloader picks up those and downloads the large media files. /downloads is mapped so it knows where to find the downloads so it can rename and move /movies is mapped so it knows my current media library, and can rename and move the file from /downloads (cache) to /movies (array) /config is the standard appdata folder /unraid is there to give it access to anything on the array Not sure what else it could besides what @jonathanm mentioned about the backup file it's creating. Not sure where it's placing that, and if it's deleting it. Nothing in the template mentions it so I don't know. I don't think it's that though because I haven't downloaded any movies today and I'm down 1GB in size. That makes me think it's either Zoneminder for the cameras, or possibly Plex DVR recording a couple TV shows? EDIT: Maybe Sonarr? Just realized there's this /dev/rtc mapped to something that I'm not sure what it even is.... Edited June 12, 2017 by Living Legend Quote Link to comment
Squid Posted June 12, 2017 Share Posted June 12, 2017 15 minutes ago, Living Legend said: Hard to say. I tend to stick to the templates. For something like Radarr, the only thing it should be downloading are .nzb and .torrent files. Then the downloader picks up those and downloads the large media files. What everyone is asking you for is to check the settings within Radarr's UI (or post screenshots). 16 minutes ago, Living Legend said: EDIT: Maybe Sonarr? Just realized there's this /dev/rtc mapped to something that I'm not sure what it even is.... Don't worry about it - its passing the date&time Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 14 minutes ago, Squid said: What everyone is asking you for is to check the settings within Radarr's UI (or post screenshots). Don't worry about it - its passing the date&time Within the UI the only path I can see that I've set is drone factory: Usually to be safe I don't type in the path, but rather click the folders and work my way down to make sure that I'm using the proper format. Quote Link to comment
Living Legend Posted June 12, 2017 Author Share Posted June 12, 2017 All other things like download clients and indexers are done with IP addresses and ports Quote Link to comment
rutherford Posted May 27, 2020 Share Posted May 27, 2020 came across this thread for docker utilization: "what's taking up all my docker.img space in there?" "Resource Monitor" in Community Applications has been replaced by cAdvisor, which is in the APPS tab. Go to SERVER:8085, click Docker Containers, at the top, scroll to bottom, and sort by Virtual Size. You can see in the image below, Krusader is hogging up 2.29GB 1 Quote Link to comment
Aerodb Posted June 20, 2021 Share Posted June 20, 2021 On 6/6/2017 at 1:17 PM, trurl said: Resource Monitor in Community Applications. does this app still exist? what name has it changed to? Quote Link to comment
Aerodb Posted June 20, 2021 Share Posted June 20, 2021 On 6/7/2017 at 4:09 PM, Squid said: 80%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img. 15%+ of the time its due to a misconfiguration of a download client where its winding up saving stuff into the docker.img due to a capitalization error on a folder (ie: /downloads is mapped, but container references /Downloads) 5% of the time its due to excessive logging by a container. Virtual Sizes AFAIK are only the sizes of the actual container, and nothing else that the container might create within the image. Net result is that its a pretty useless metric, but since the docker system has it available, I make it visible. You can peruse the Temporary Docker FAQ for some suggestions, or the investigation regarding out of control apps logging "Hi! I'm still running. How are you doing?" every second here:http://lime-technology.com/forum/index.php?topic=45249.0 so I have confirmed my docker mapping, and I have log sizes limited and have confirmed they are not the issue. I'm pretty sure I'm in the 80% block. but i have a 100gb docker image file and I'm at 71% full now. Any advice on how I can root out what is causing this issue? Quote Link to comment
Squid Posted June 20, 2021 Share Posted June 20, 2021 9 minutes ago, Aerodb said: does this app still exist? what name has it changed to? No. You can get the container sizes right from the docker tab (Container Size button) to see which app is taking up which space (not quite 100% accurate, but will give a good idea) 1 3 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.