April 16, 20197 yr 46 minutes ago, DerfMcDoogal said: Thank you! This solved my issue. I was doing a lot of operations in Plex which I believe is causing all sorts of logging. I got the warning about 71% usage and by the time I went to investigate it was at 78% and by the time I found the thread and read to page 2 it was 97% and I had to stop all of my containers. This immediately dropped my usage down to 28%. All I can think is that Plex is logging and that those logs are normally trimmed every night but since I was doing a lot of operations it spun out of control before the log could be trimmed? No idea. It was increasing so fast I didn't have time to mess around. FYI, that command won't "solve" anything, but rather delays it from happening again. The solution is this https://forums.unraid.net/topic/57181-real-docker-faq/#comment-564326
April 16, 20197 yr 2 hours ago, Squid said: FYI, that command won't "solve" anything, but rather delays it from happening again. The solution is this https://forums.unraid.net/topic/57181-real-docker-faq/#comment-564326 Done. Thanks!
June 19, 20197 yr On 4/11/2018 at 10:23 AM, vizi95 said: I had the same issue. It has to do with docker log file sizes. Just run this and your utilization will come down a lot: truncate -s 0 /var/lib/docker/containers/*/*-json.log It's safe to use while the docker is still running. Aw man, this saved me. I had a docker with a port binding issue, I guess it was filling logs constantly, but after repairing the bind problem, i didn't know what was eating up the docker image file... logs. *shakes fists into air*
June 19, 20197 yr Aw man, this saved me. I had a docker with a port binding issue, I guess it was filling logs constantly, but after repairing the bind problem, i didn't know what was eating up the docker image file... logs. *shakes fists into air*Good to see you back PhAZeBut, you are better off adding the settings into extra params or nuking the image and setting up log rotationThe commands listed is a temporary solutionSent from my NSA monitored device
June 20, 20197 yr Don't know if this will help or not. I had 85% usage a week back. After looking around I found it was Plex sync. My wife had friend's Blu-ray complete series synced to her tablet. Once I deleted the synced files everything returned to normal. I think maybe it caused problems because her tablet storage was full. I honestly don't know. It also filled up my cache drive in the process.Sent from my Moto Z (2) using Tapatalk
June 26, 20197 yr On 6/19/2019 at 1:37 PM, Squid said: Good to see you back PhAZe But, you are better off adding the settings into extra params or nuking the image and setting up log rotation The commands listed is a temporary solution Sent from my NSA monitored device Yea, this was more to prove that I had the correct docker as the culprit. I just nuked the image and set an extra param path.
June 26, 20197 yr Just now, PhAzE said: Yea, this was more to prove that I had the correct docker as the culprit. I just nuked the image and set an extra param path. docker is weird with log rotations. The global setting (Settings - docker) only takes effect on new installs. Any existing installs aren't affected and need the extra params. It may be academic now, but by nuking docker.img, all the installs became "new", and wouldn't need the extra param if you had set up the global. There was a debate on GitHub about how to handle things because it isn't anywhere near intuitive how the system works on logs.
September 27, 20196 yr On 9/6/2017 at 8:44 AM, tillkrueger said: my thought exactly, Squid...since I have no other way of knowing what Krusader is doing...it's a really cool file manager, I must say! if the UI could be configured to always show the Queue at the bottom and the file manager at the top, that would make it an over the top solution for my file management duties on the server...if I were better at using rsync in Terminal, maybe I'd feel different about that, but for a GUI guy like me, Krusader is pretty great...sure beats Dolphin, in my mind. @tillkrueger I've used Dolphin on Kubuntu and find it very functional. Can you tell me how Krusader is better as I am interested in knowing what features I'm missing out on.
September 27, 20196 yr 37 minutes ago, frakman1 said: @tillkrueger I've used Dolphin on Kubuntu and find it very functional. Can you tell me how Krusader is better as I am interested in knowing what features I'm missing out on. I also used Dolphin before I started using Krusader, and felt much more drawn to the UI of Krusader...just seemed more refined and “tight” to me. And when I found binhex Krusader, that impression was further enhanced by binhex’s addition of the windows bar at the bottom, which allows direct access to all active windows of your Krusader instance, like the queue, the directory sizes tool, and everything else Krusader can do in parallel to being the excellent file manager that it is.
September 28, 20196 yr @tillkrueger I played around with it and I have some questions: 1- Where is the left side panel that has all the bookmarks for various commonly used folders? I don't see a way to add that. 2- Under Settings->Show command line. What does that actually do/allow you to do?
September 28, 20196 yr Hmm, to be honest, I didn’t know that there is a bookmarks panel for frequently used folders, so I couldn’t tell you. Never used the command line feature either, so I am sorry to say that I can’t be of any help with either one of your questions. But someone here surely must know.
October 25, 20196 yr Hi, My situation is a bit different but it fits in this topic I guess. I have a Nextcloud setup with a Letsencrypt reverse proxy. When I upload a large file to Nextcloud my docker image fills up. As far as I can tell the Letsencrypt container has the most data transfered but I'm unable to find out where exactly that temporary data is stored. All I could find is the /dev/loop2 gets full and obviously I get an error in Nextcloud. Any hint is apreciated.
October 25, 20196 yr This is the config server { listen 80; server_name ..... return 301 https://$server_name$request_uri; } server { listen 443 ssl; server_name ..... root /config/www; index index.html index.htm index.php; ###SSL Certificates ssl_certificate /config/keys/letsencrypt/fullchain.pem; ssl_certificate_key /config/keys/letsencrypt/privkey.pem; ###Diffie–Hellman key exchange ### ssl_dhparam /config/nginx/dhparams.pem; ###SSL Ciphers ssl_ciphers ..... ###Extra Settings### ssl_prefer_server_ciphers on; #ssl_session_cache shared:SSL:10m; ### Add HTTP Strict Transport Security ### add_header Strict-Transport-Security "max-age=63072000; includeSubdomains"; add_header Front-End-Https on; client_max_body_size 0; location / { proxy_pass ..... proxy_buffering off; } location = /.well-known/carddav { return 301 $scheme://$host/remote.php/dav; } location = /.well-known/caldav { return 301 $scheme://$host/remote.php/dav; } add_header Referrer-Policy no-referrer always; } I have proxy buffering off so I could avoid this problem.
October 25, 20196 yr Not quite, look at your disk and directory mappings in the Docker settings for NextCloud.
October 25, 20196 yr Sorry I didn't provide enough details. A kind of buffering happens. Is not related to volume binding. If the file is smaller then my docker image free space, after the upload ends, the file goes on the array at the correct destination. BUT if the file size exceeds my docker image free space, the dicker image gets 100% full, the upload fails and the docker image gets back to normal. So I need to find out where this cache exists to bind it outside of docker image. Here are my mappings. /localdata is created by me for external storage in Nextcloud. It works fine. /data -> /mnt/user/DATA/Cloud/ /localdata -> /mnt/user/ /config -> /mnt/user/appdata/nextcloud Actually this happens only if I upload by shared link and not if I'm logged in as a user. I'll start a new topic on Nextcloud thread I think. Edited October 25, 20196 yr by andreidelait New discovery
October 25, 20196 yr 2 hours ago, andreidelait said: Sorry I didn't provide enough details. A kind of buffering happens. Is not related to volume binding. If the file is smaller then my docker image free space, after the upload ends, the file goes on the array at the correct destination. BUT if the file size exceeds my docker image free space, the dicker image gets 100% full, the upload fails and the docker image gets back to normal. So I need to find out where this cache exists to bind it outside of docker image. Here are my mappings. /localdata is created by me for external storage in Nextcloud. It works fine. /data -> /mnt/user/DATA/Cloud/ /localdata -> /mnt/user/ /config -> /mnt/user/appdata/nextcloud Actually this happens only if I upload by shared link and not if I'm logged in as a user. I'll start a new topic on Nextcloud thread I think. Mapping to /mnt/user might well be a problem -you should also have a directory at the end of that path (perhaps something like ‘/mnt/user/appdata/nextcloud/localdata’). Edited October 25, 20196 yr by itimpi
February 10, 20206 yr My docker image has also been increasing for a while. This is what I am seeing based on the prior thread suggestions: root@baconator:~# du -h -d 1 /var/lib/docker/ 1.2M /var/lib/docker/containerd 93M /var/lib/docker/containers 0 /var/lib/docker/plugins 99G /var/lib/docker/btrfs 42M /var/lib/docker/image 120M /var/lib/docker/volumes 0 /var/lib/docker/trust 208K /var/lib/docker/network 0 /var/lib/docker/swarm 16K /var/lib/docker/builder 56K /var/lib/docker/buildkit 1.4M /var/lib/docker/unraid 0 /var/lib/docker/tmp 0 /var/lib/docker/runtimes 99G /var/lib/docker/ 99GB docker file?? lol My docker is supposed to be 30GB (see below) I also performed a scrub as well. I dont think it is log files: Just like everyone else, nothing in these show anything anywhere near 30GB. I have tried to truncate logs and that didnt do much either by using the following: truncate -s 0 /var/lib/docker/containers/*/*-json.log I did see this in Cadvisor though: Any ideas? This is highly frustrating. Thanks for any help. Edited February 10, 20206 yr by danktankk added new data
February 10, 20206 yr 55 minutes ago, danktankk said: My docker image has also been increasing for a while. This is what I am seeing based on the prior thread suggestions: root@baconator:~# du -h -d 1 /var/lib/docker/ 1.2M /var/lib/docker/containerd 93M /var/lib/docker/containers 0 /var/lib/docker/plugins 99G /var/lib/docker/btrfs 42M /var/lib/docker/image 120M /var/lib/docker/volumes 0 /var/lib/docker/trust 208K /var/lib/docker/network 0 /var/lib/docker/swarm 16K /var/lib/docker/builder 56K /var/lib/docker/buildkit 1.4M /var/lib/docker/unraid 0 /var/lib/docker/tmp 0 /var/lib/docker/runtimes 99G /var/lib/docker/ 99GB docker file?? lol My docker is supposed to be 30GB (see below) I also performed a scrub as well. I dont think it is log files: Just like everyone else, nothing in these show anything anywhere near 30GB. I have tried to truncate logs and that didnt do much either by using the following: truncate -s 0 /var/lib/docker/containers/*/*-json.log I did see this in Cadvisor though: Any ideas? This is highly frustrating. Thanks for any help. Try this user script https://forums.unraid.net/topic/48707-additional-scripts-for-userscripts-plugin/page/8/?tab=comments#comment-682421 It calculates the sizes slightly different than the Container Size button. Is there a significant difference? Note that the script can only calculate sizes for running containers.
February 11, 20206 yr 44 minutes ago, Squid said: Try this user script https://forums.unraid.net/topic/48707-additional-scripts-for-userscripts-plugin/page/8/?tab=comments#comment-682421 It calculates the sizes slightly different than the Container Size button. Is there a significant difference? Note that the script can only calculate sizes for running containers. Hey, Thanks for the reply. I dont see anything here that would indicate an issue either. This is what I got from running the script: Edited February 11, 20206 yr by danktankk
February 11, 20206 yr 9 hours ago, danktankk said: My docker image has also been increasing for a while. This is what I am seeing based on the prior thread suggestions: root@baconator:~# du -h -d 1 /var/lib/docker/ 1.2M /var/lib/docker/containerd 93M /var/lib/docker/containers 0 /var/lib/docker/plugins 99G /var/lib/docker/btrfs 42M /var/lib/docker/image 120M /var/lib/docker/volumes 0 /var/lib/docker/trust 208K /var/lib/docker/network 0 /var/lib/docker/swarm 16K /var/lib/docker/builder 56K /var/lib/docker/buildkit 1.4M /var/lib/docker/unraid 0 /var/lib/docker/tmp 0 /var/lib/docker/runtimes 99G /var/lib/docker/ 99GB docker file?? lol My docker is supposed to be 30GB (see below) I also performed a scrub as well. I dont think it is log files: Just like everyone else, nothing in these show anything anywhere near 30GB. I have tried to truncate logs and that didnt do much either by using the following: truncate -s 0 /var/lib/docker/containers/*/*-json.log I did see this in Cadvisor though: Any ideas? This is highly frustrating. Thanks for any help. du command goes across filesystem for sure I've tried this one too and mine returns a 74G /var/lib/docker/btrfs, my docker image having 40G. However if I insert -x argument it returns 0 for that volume. I'm sure it calculates some volume mappings or something. So usually I use root@Tower:~# docker system df TYPE TOTAL ACTIVE SIZE RECLAIMABLE Images 28 28 11.57GB 784.9MB (6%) Containers 28 16 1.744GB 372.3MB (21%) Local Volumes 11 1 39.68MB 0B (0%) Build Cache 0 0 0B 0B or root@Tower:~# df -h | grep /dev/loop2 /dev/loop2 40G 15G 25G 38% /var/lib/docker Edited February 11, 20206 yr by andreidelait
February 11, 20206 yr My docker size is still an issue. This should not be that difficult to figure out. lol If there was a new feature to be requested for unraid, it would be a new mechanism to show precisely what docker containers have what and where the space usage is increasing. In windows this is extremely easy to do. Thanks again for the help thus far. Edited February 11, 20206 yr by danktankk
February 11, 20206 yr Try this: go to the Shares tab. Click on the folder icon next to the system share. Now click on the folder for Docker. What size does it report on that page for the docker image size? Also, when you scrubbed your container, did you notice the amount scrubbed was only 22GiB? If you're not familiar with how Docker, volume mappings, thin provisioning, etc. work, the du command can certainly confuse you.
February 12, 20206 yr @jonp Thank you for the reply! This is the amount of space it is currently reporting: All containeras added up is 15.6GB or so: however, I am still seeing this usage and getting notifications often: If the amount scrubbed is 22.16GB, it does make sense that it would report, and does report 75-80% usage. Is there any reason for this discrepancy (15.6GB vs 22.1GB you can think of? Thanks again for the input. Edited February 12, 20206 yr by danktankk
February 12, 20206 yr There are probably some additional files inside the docker.img or perhaps images that aren't active in use but are consuming a tiny bit of storage. I really wouldn't be concerned.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.