Help with Docker High Image Disk Utilization


maxse

Recommended Posts

Hey guys,

So after setting things up in unraid 6 and having everything running smooth for the pas 30 days I just got this message:

"Docker High Image Disk Utilization 88%"

The only dockers enabled and running for the past 30 days have been, plex, sab, radarr, sonarr. This message never came up before so I'm a bit confused as to what it means. As far as I know my paths have been correct, and it's been running for 30 days trouble free, if something was wrong wouldn't it show up before or is it something that I just need to make the space bigger since they are running longer?

 

Not sure what I need to be doing if anything? Or if I did something wrong? I used to run unraid 5 for years without really even checking anything and it's been quite frustrating setting everything up in unraid 6 with docker, but you guys are awesome and helped me with everything!. I am attaching the diagnostics file with this post. Thanks so much!

 

Figured listing my mappings would be helpful (using binhex for all except plex which ioserver:

 

SAB:

/data --> /mnt/user/downloads

within sab:

Temp is set to  /data/incomplete

Completed folder is set to /data/complete

 

Plex Docker: /transcode --> /mnt/user/downloads/transcode

within plex transcode folder is set to /transcode

 

 

 

sabertooth-diagnostics-20190207-2254.zip

Edited by maxse
Link to comment

yes, everything starts with a '/'

 

So I had a bunch of dockers that had updates that were ready that I did not do. I just hate updating something when it works fine.. I did it the updates and everything has been running for about 12 hours now and it is staying at 32-33% usage which is what it started with 12 hours ago. I'm not sure what could have happened. 

 

Container sizes look well below the 2 gigs. Maybe 5 gigs total. I probably should have checked this last night but I wasn't aware of this menu. Could it have spiked up like that because of updates that were ready and i didn't do? Is that normal behavior? I just want to make sure that this is not something that keeps happening in the future.

 

When adding up the container size  the Log files add up to about 66MBs, Writable 100MBs, and Container about 6GBs

Edited by maxse
Link to comment
  • 1 year later...

All,

 

I am having the same problem.

 

I just installed Deluge and got it working.

 

I am getting these warnings:

image.png.2955cf7539c9940b4974f0654a9139e3.png

 

I did a docker container size and got this:

image.png.31ff47ccbd720ca686cff17bac45314c.png

 

Did I mess up the setup?

 

I did as was suggested that I use the cache drive and mover.

 

The disk utilization looks to come from writeable/container = 72%

 

I have to admit since I just starting using it I really don't know that much about it ....

 

Any tips on a direction would be useful.

 

Thanks in Advance

 

Cheers

Link to comment

Not the cause of this problem, but your system share has files on the array. In fact, you have it set to move files to the array.

 

You want appdata, domains, and system shares to be all on cache and none on array so your docker and VM performance won't be impacted by the slower parity writes, and so they won't keep array disks spinning.

 

Do you actually have any VMs?

Link to comment

Yes, noticed it when I looked to see how the server utilization was. It was much higher than normal. 

 

I only have 1 vm - Pf sense. 

 

This is the binhex container with vpn. 

 

I will try this tomorrow as it is quite late. 

 

Thanks for your feedback and I'll report back after I give this a go. 

 

Cheers

 

Link to comment

trurl,

 

I hope I understood what you were asking.

 

I changed the deluge docker to debug and the following was the output from the run command:

image.thumb.png.6d8b81f31c0e4445711991b971a2de25.png

 

Let me know if this is what you were looking for and/or do you need something else???

 

Oh I made the changes to the shares as you suggested.

 

Thanks again

 

Cheers

 

Deluge_log.txt

Edited by toolmanz
Link to comment

Also,

 

I ran fix common problems and I get this message:

 

"Share system set to cache-only, but files / folders exist on the array You should change the share's settings appropriately

or use the dolphin / krusader docker applications to move the offending files accordingly. Note that there are some valid use cases for a set up like this. In particular: THIS"

 

How to find the offending files? and/or will mover move these out of the array?

 

I am doing some research on this and it seems a fairly common problem .... Ill chase this down.

 

 

Cheers

Edited by toolmanz
addl info
Link to comment
3 hours ago, toolmanz said:

I am doing some research on this and it seems a fairly common problem .... Ill chase this down.

I've certainly explained it many times.

 

Mover ignores cache-no and cache-only shares.

Mover moves cache-yes shares from cache to array.

Mover moves cache-prefer shares from array to cache. This is the one you want.

Mover cannot move open files, so if Docker or VM services are enabled (in Settings - Docker and Settings - VM Manager) it won't be able to move your system files.

Mover will not overwrite duplicates when it moves. Sometimes this will require manual cleanup.

Link to comment

OK, back to your docker run. I don't use this docker, but that looks like it might be OK. The other piece of this puzzle though is the paths you configure within the application. If the application is writing to a path that isn't mapped, it is writing into the docker image. The paths the application uses must exactly match (Linux is case-sensitive) the container paths in the docker mappings. And only absolute paths (beginning with /).

Link to comment

trurl,

 

Well I went ahead and did the search and found what the problem was.

 

You were exactly right! 👍

 

The docker.img file was totally on the array, and libvert.img was on the array and the cache.

 

I have now moved both to cache and have seen the disk utilization of the cache go up by the requisite amount.

 

I actually shutdown the dockers and vm and manually copied the files elsewhere, deleted and copied it back.

 

48 minutes ago, trurl said:

Mover ignores cache-no and cache-only shares.

Mover moves cache-yes shares from cache to array.

Mover moves cache-prefer shares from array to cache. This is the one you want.

Mover cannot move open files, so if Docker or VM services are enabled (in Settings - Docker and Settings - VM Manager) it won't be able to move your system files.

Mover will not overwrite duplicates when it moves. Sometimes this will require manual cleanup.

 

Nice to know there is a better way to do this using the mover app!

 

I'm gonna put your rules in a frame and mount it on the wall! You explained it perfectly. 😁. You may want to "sticky" these rules!

 

I have left the system share, along with appdata and domains  as cache-only which I'm hoping will keep things right.

 

Well, a lot of lessons learned and a much more stable system thanks to you - you are the man! 😎

 

By the way - system is floating between 2% - 5% cpu usage - I'm sure the server has a happy face 😉

 

Thanks again

 

Cheers

 

 

Link to comment
  • 5 weeks later...
9 hours ago, trurl said:

Go to Settings - Docker and stop docker. Set the slider at upper right on that page to Advanced View. Turn on log rotation. Enable docker again.

 

Which dockers do you run?

If log size is the issue, that won't work.  You would need to delete the image to get that to work.  See  https://forums.unraid.net/topic/57181-docker-faq/#comment-564326

 

 

@tmoran000  Post a screenshot of the popup that happens when you click Container Size on the Docker Tab

Link to comment

You will have to delete that docker image as @Squid suggested to get it fixed, even though log size doesn't seem to be the issue. But you need to figure out what you have done wrong to make this happen.

 

Any idea why your plex container is so large? You are almost certainly writing to a path that isn't mapped. With plex, about the only writing it is going to do is transcoding, and recording. For comparison, my plex container size is only 723MB.

 

Krusader, the second on the list, is known to be somewhat large so that is probably fine.

 

Post your docker run command for plex as explained in this very first link in the Docker FAQ:

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.