Expand Docker img Size

[crazycam425]

Hi can anyone please tell me how to increase my docker img size without losing anything. I feel like this should be simple but I cannot find a thread anywhere explaining this. Thank you for the help!

[BRiT]

From the Docker FAQ 

 

 

[steve1977]

Does this still work in the same way for 6.4.1?

 

I disabled the docker and set to advanced view. But I don't see an option to change the size?

[Squid]

Yes it's still there.   Docker vdisk size

[devilwarriors]

For anyone looking for it you need to check the advanced view checkbox thing on the right to see those settings.

[trurl]

Increasing size of docker.img is usually not the fix if you are filling it. Making it larger will only make it take longer to fill. The usual reason for filling is an application writing to a path that isn't mapped. 

[DaveW42]

A quick note.   I followed the instructions above, but the option to change the vdisk size never appeared.  There must be some conditions that prevent the option from appearing (perhaps if your percentage of docker image utilization is too high???).  I removed  a couple of docker applications and rebooted my unraid machine, and then the option appeared and I was able to change the docker image size.

[John_M]

19 minutes ago, DaveW42 said:

There must be some conditions that prevent the option from appearing

 

Not high utilisation, but if the image is in use its size can't be changed. Perhaps it couldn't be unmounted because there was an open file on it. Rebooting would have fixed that problem.

 

[trurl]

14 hours ago, DaveW42 said:

change the docker image size

 

What did you change it to? 20G should be more than enough.

 

On 12/20/2020 at 12:02 PM, trurl said:

Increasing size of docker.img is usually not the fix if you are filling it. Making it larger will only make it take longer to fill. The usual reason for filling is an application writing to a path that isn't mapped. 

 

Go to Tools - Diagnostics and attach the complete Diagnostics ZIP file to your NEXT post in this thread.

 

[DaveW42]

Thanks, John_M and trurl.   I realize that it was a band-aid solution, but I changed my docker size from 20gb to roughly 80gb.  In doing so--and recognizing the sound advice in this thread--I knew that what I really needed to do was to identify what the problem was ... the trouble was that I didn't want the image to keep filling up while I was trying to identify the problem.  In short, I was just trying to buy myself a little time.   I have about 16 dockers installed, although usually only about 7 of them are running at any given time.  

 

trurl, I suspect the problem concerns how I configured my roonserver docker.  Fix Common Problems identified two directories that differed only in case (Music vs. music).  Additionally, I suspect that awhile back roon had been saving backups to the container rather than externally.   Those backups are unfortunately all corrupt at this point, likely due to the above issue.  I think I need to do some further investigation before reporting back, or I will just be wasting your time.

[trurl]

6 hours ago, DaveW42 said:

Fix Common Problems identified two directories that differed only in case (Music vs. music)

Linux is case-sensitive. If an application is using a path different from the mappings, that path is inside docker.img

 

6 hours ago, DaveW42 said:

 I have about 16 dockers installed, although usually only about 7 of them are running at any given time.  

I have 18 dockers running at the same time, and they use less than half of 20G docker.img

[trurl]

7 hours ago, DaveW42 said:

 I have about 16 dockers installed, although usually only about 7 of them are running at any given time. 

The space in docker.img is mostly just the downloaded executables for each of the containers, so if they are installed it doesn't really matter how many are currently running they are still taking space.

[DaveW42]

Thanks for your insight, trurl!  I was aware of the case-sensitive nature of linux, but just made a mistake at the time.  Actually, I wonder if the container template even forced that particular directory ("music") to appear  ... for example, I see a typo in the Roon docker that labels "Backup" as "Backip" ...  I've tried to just remove the "Backip" from the template when creating the docker container, but it keeps reappearing even after I delete it.  Also, with regard to paths, can I just create whatever paths in a docker container that I want, as long as I map them properly OUTSIDE the container? 

 

Is there a particular book, website, or post you would recommend to read, to have a basic understanding concerning container paths, dockers, and perhaps even the linux file system?  I don't have any illusions that I will be able to learn how to program in linux, but it sure would be nice for me to have a basic understanding of how directories are organized, the rules associated with how docker containers are configured, etc. so I can find things and be generally familiar with how to navigate around and 'follow the rules.'   My knowledge right now feels pretty fragmented.  I learn a lot from SpaceInvader One's videos, but there are a lot of gaps in my knowledge.  For example, it caught me off guard at one point that using "/Music" as a path in docker could lead to a different outcome than specifying, for example, just "Music." 

 

Thanks again for your help!

 

[ChatNoir]

You might start by reading this :

 

 

[DaveW42]

Thanks, ChatNoir!  

 

I had read portions of that thread before ... each time I reread these things one more piece of information sticks.  Within the link you shared, the post i have referenced below is particularly informative.   

 

To provide an additional example of where I struggle, consider within the post below the reference to directories such as "/mnt/user"  etc.   Where do I actually see these directories in the Unraid GUI?  I can gain access to them if I open a terminal and navigate to them via the command line (e.g., "cd /mnt"), but are they accessible somewhere else too?   "/mnt" seems to be a pretty darn important directory (I see elsewhere that the disks are all 'mounted' to this directory).   And if I go into a file manager program like CloudCommander, for example, I can navigate to  "/mnt"  but then no subdirectories appear (i.e., no "/mnt/user").  Why is this the case?   Is "/mnt/user"  a hidden directory?  The perceived inconsistencies here make it harder to figure things out and identify the basic principles and rules by which the Unraid file system operates. 

 

Having said all this, I really really do like Unraid quite a bit.  I just wish I know more about what was going on, so I could better troubleshoot any issues as they occur.  Also, I think Squid is absolutely incredible in terms of his contributions to this community (not trying to criticize the FAQ in any way, just trying to share where my ignorance lies so I can become more knowledgeable).

 

 

 

 

[ChatNoir]

That's the tricky part with Dockers and it's configuration.  
I am no expert yet but here is what I understood.

 

On one side you have the Unraid system with :

/mnt/user/cache

/mnt/user/Media

/mnt/user/photos

/mnt/user/appdata

etc. whatever YOUR shares are in Unraid. And also all the stuff under the hood that you might not want to play with.

 

On the other side you have each specific Docker container that is an isolated Linux system. You have all the regular Linux directories but by default, you are stuck within the container and cannot see the stuff in Unraid or the other containers.

There will be :

/bin

/boot

/mnt

/sys

etc. whatever is needed in the Container. But we don't really care about /mnt within the container.

 

Now that's enough for some containers, but most of the time, you will need to map paths from the container to a path of the host Unraid in our case.

A few examples :

/Media ==> /mnt/user/Media

/Series ==> /mnt/user/Media/Series

/Ireland ==> /mnt/user/photos/2011/August/Ireland

 

So in you example of Cloudcommander, you would have to map the directories from Unraid you want to interact to with directories inside the container.

Say you want to move images from one share to another for compression or whatever, you could add paths :

 

/tobemoved ==> /mnt/user/Pictures/Source

/compression_input ==> /mnt/user/Randomcompression_SW/input

/compression_ouput ==> /mnt/user/Randomcompression_SW/output

/final_storage ==> /mnt/user/Pictures/Compressed_for_storage

 

I hope this helps a bit.

[trurl]

6 hours ago, DaveW42 said:

directories such as "/mnt/user"  etc.   Where do I actually see these directories in the Unraid GUI?

The User Shares are in /mnt/user. If you have a User Share named Music, it is at /mnt/user/Music. There are also the disk paths, /mnt/disk1, /mnt/disk2, ..., /mnt/cache, and possibly others if you have other "pools" in 6.9+. And there are the /mnt/disks and /mnt/remotes created by Unassigned Devices.

 

The User Shares are simply the aggregate of all top level folders on the array and pools. A folder named for the share is created at the top level of the disks as needed according to the settings for the share. Conversely, any folder at the top level is automatically a share named for the folder, whether or not you created it as a User Share in the webUI. Any share you haven't made settings for will have default settings.

 

Since the user shares are just another view of the disks, you must not mix disks and user shares when moving / copying, because Linux doesn't know that the source and the destination might be the same file and it will try to overwrite what it is trying to read.

 

 

 

 

[DaveW42]

Wow, thanks so much ChatNoir and trurl!!!  I've read and re-read both of your posts several times, and things are sinking in a bit more each time. 

 

ChatNoir, I liked your use of color to denote the Unraid vs. Docker side of things.  That helps quite a bit.  Interesting that the Linux file structure appears on both sides (on Unraid side and then in Docker nested within UnRaid).  trurl, your use of the term 'view' in describing shares definitely provided an epiphany of sorts, and the clarification concerning disks and shares and the extra directories created by Unassigned Devices was extremely helpful. 

 

So, in essence we have an underlying file structure, and one can look at that structure through multiple perspectives such as Unraid's attached physical disks ("disks"), particular directories the user has chosen to make available ("shares")  under particular constrains (i.e., permissions and configurations such as "private" and "public"), directories located on other computers that we want easy access to ("remotes"), and  any "top level" folders on the disk.   If I am reading this correctly--based on ChatNoir's comment concerning "/mnt within the container"--if I am navigating through directories in Unraid (i.e., not a docker), I should be able to find the files I create within "/mnt", with "/mnt/user" being particularly important (thanks trurl!).  If I am in a Docker container, we only see what is directly inside the container and what we have specifically configured to pass through the container.  Thus looking at "/mnt" is essentially useless, since I will likely want any file I create to go into one of the Unraid paths that I have mapped to appear in the container (i.e, so the docker doesn't get filled up, and also so I don't lose a file I have created if I later need to rebuild the docker image).   Putting stuff in "/mnt" in a docker container would be a quick way to fill up the docker image, and a quick way to lose files if something goes wrong with the container.

 

Thanks so much!!!  This really explains why it always felt like trial and error to find a file I was interested in.  In short, views on files are constrained when looking through the lens of a docker container.  On the other hand, when you are looking at the Unraid file structure, there can be many ways to access the same file, depending on which "view" you choose to pursue.  That is totally cool.  But also a bit confusing!  I think I am starting to put things together a bit.

 

I'll have to look further into the concept of a "pool."  I haven't seen that one before and will need to look into it to understand.

 

Thanks again, and also sorry it took me a few days to respond (life called ...)

 

Dave

[ChatNoir]

Glad we were able to help.  

 

The Pool concept is simply a disk or group of disks managed directly by Unraid that is NOT the Array.

 

Before 6.9, this wording was not much used as the only pool was the cache.

Since 6.9, you can create many pools depending on your use case.

 

Single disks can be XFS or BTRFS, but multi disks pools need to be BTRFS and can then be either on various RAID mode or in "Single". The default for multi disk is RAID1.

[DaveW42]

Thanks, ChatNoir!  I'm still on 6.8.2, so I'll need to wait on exploring that one until I have time to make sure I can upgrade to 6.9 safely.  In short, I see from a spaceinvaderone video that docker container updates are recommended when I make the upgrade.  I will want to make sure that--when I originally created those containers--I didn't inadvertently violate any of the guidance you and trurl provided above.  Just want to make extra sure I don't lose anything.

 

Thanks again!

Dave 

[fespinoza831]

Hello, can someone help me identify what is filling up my docker image? and maybe show me how I can check my self in the future.

plexserver-diagnostics-20221006-0844.zip

[trurl]

Go to Dockers page, click Container Size button at bottom, post a screenshot

[fespinoza831]

Here it is.

[trurl]

That looks OK and docker.img wasn't full in those diagnostics. Is it filling or just using more than you expect? The usual cause of filling docker.img is an application writing to a path that isn't mapped.

[fespinoza831]

ok thanks, I was scared that it might be filling up since i had to expand it to 30gb at one point.