[6.12.8] Understanding System Overview and percentages

[stainless_steve]

I always thought I understood how the on the dashboard the system overview is to read or at least did not give it much thought. But now I have troubles as my understanding seems to be wrong.

 

As you can see I have 64GB of RAM and around 5% are used, which should be around 3.2GB. I also think, that from the FLASH drive´s 16GB 4% are used.

• Where is the maximum size of 128GB coming from?
• what determines the maximum size of the Log ?
• and most importantly: why is the docker at 79% (732GB)? I checked and the docker.img file is just 21.5GB big and I have plenty of space available. It seems that the maximum is about 1TB but where does it referre to? It seems I am running out of spce and I don´t know why.

 

Can somebody explain and help me please? Thank you

 

homeserver-diagnostics-20240415-2053.zip

[JonathanM]

6 hours ago, stainless_steve said:

Where is the maximum size of 128GB coming from?

Motherboard reported, may be accurate, maybe not.

 

6 hours ago, stainless_steve said:

what determines the maximum size of the Log ?

fixed allocation by Unraid

 

6 hours ago, stainless_steve said:

and most importantly: why is the docker at 79% (732GB)? I checked and the docker.img file is just 21.5GB

It's the percentage of space used INSIDE the docker.img file. Not the free space on the drive holding the image file.

[stainless_steve]

3 hours ago, JonathanM said:

It's the percentage of space used INSIDE the docker.img file. Not the free space on the drive holding the image file.

Ok thanks. Unfortunately I don't understand this. Is this something I can change and eliminate this limitation?

[itimpi]

1 hour ago, stainless_steve said:

Is this something I can change and eliminate this limitation?

It is not a limitation - it is simply a report.   If you run out of space within the image then docker will start misbehaving , although the default size of 20GB is more than enough for the vast majority of users.   If while running containers the usage increases and you run out of space this typically means a misconfigured docker container.

 

You can control the size allocated to the image file under Settings->Docker.   You can also a directory rather than an image file at all although this is not recommended.

[stainless_steve]

Please excuse me, but I am still quite new to Unraid and I am trying to learn.

 

The only container that could build up this much, is deluge I think. But as you can see in the screenshot, data should be stored outside the image.

But this is set up following specific YT guides...

So config is in appdata on the cache and all the downloads should be on the data share.

Could it have something to do with the settings in deluge?

I don´t see where, because as I understand it, "torrents" is located under /mnt/user/data/... and not under config as I could be, I assume, if it is like default and this post in the FAQ suggests:

 

Besides hosting my movies with jellyfin, I do not much with my server.

 

 

appdata is the only one of my shares that is on the cache, and it takes up 1.27GB...

Can you see and tell me what setting is wrong and should be changed in your opinion?

 

thanks in advance!

[JonathanM]

I don't understand what you think is wrong, everything looks good to me.

[stainless_steve]

I think/thought that I will run into a problem, when Docker reaches 100% (see first post). It seems to fill up when downloading.

I am at 80% (737GB) but I have 4.5 TB free space left on the array.

I am unsure what will happen?! I thought I will run into troubles as I did when previously my cache was full. It was also docker, but then I started from scratch and it was just quite some work to get everything back running as I like...

[JonathanM]

39 minutes ago, stainless_steve said:

I am at 80% (737GB)

80% is about 16GB, not 737GB

On 4/15/2024 at 3:22 PM, stainless_steve said:

I checked and the docker.img file is just 21.5GB big

 

20 hours ago, JonathanM said:

It's the percentage of space used INSIDE the docker.img file. Not the free space on the drive holding the image file.

 

[ConnerVT]

Reading this thread, I think it's time for a quick Docker tutorial:

 

-- the docker.img file (which typically resides in the Unraid system share on the cache pool drive, by default) is where the docker "programs" reside.

-- each docker program's configuration files, logs, databases, and other support files are typically stored in the docker's appdata, which usually is in a folder in the appdata share (usually on the cache pool drive, for fast accesses).  This is typically mapped from inside the docker container to the appdata share folder by a Container Path entry in the docker's template.

-- For dockers which manage a large amount of data (media servers, photo programs, torrent/NZB downloaders, etc.) data is typically stored on the Unraid array, and the docker has a path mapping from inside the docker container to where the data is in the array.

 

In Unraid's Dashboard System tile, the Docker line displays how much space the docker programs are using of the allocated docker.img file (Unraid's default size for this file is 20GB).  Ideally, the only time this size should change is when you install an additional docker program.  However, it is good to monitor this size, as sometimes things mess up, and require you take some action.  Some possible things that happen are:

 

-- Improper Container Path mapping - Say a download program does not have a Path mapped outside the docker container.  Downloads will be written to the docker.img file instead, quickly filling it and causing all of your dockers to crash.

-- Some containers keep their log files with their program in the docker.img.  Check if there is configuration setting in the program to rotate out old logs, and avoid using debug logging if unneeded.

-- Almost always avoid updating a docker from within the docker, better to update the entire container.  Some programs store backups when updated, which will likely end up in docker.img.

 

I have 37 Docker containers installed in my server.  25 of them running all of the time.  My docker.img file is at 22.9GB used.

[stainless_steve]

This is what I am trying to tell you in my first post:

Quote

On 4/15/2024 at 9:22 PM, stainless_steve said:

and most importantly: why is the docker at 79% (732GB)? I checked and the docker.img file is just 21.5GB big and I have plenty of space available. It seems that the maximum is about 1TB but where does it referre to? It seems I am running out of spce and I don´t know why

 

 

The docker.img is small as it should be according to the "file explorer" (don´t know the proper name):

If I have wrongly mapped the data storage into the containers itself, from my understanding the docker image itself (or atleast the "system" share) should grow in size. But it doesn´t do that.

Quote

7 hours ago, ConnerVT said:

I have 37 Docker containers installed in my server.  25 of them running all of the time.  My docker.img file is at 22.9GB used.

 

same as mine...

 

Quote

7 hours ago, ConnerVT said:

-- Improper Container Path mapping - Say a download program does not have a Path mapped outside the docker container.  Downloads will be written to the docker.img file instead, quickly filling it and causing all of your dockers to crash.

-- Some containers keep their log files with their program in the docker.img.  Check if there is configuration setting in the program to rotate out old logs, and avoid using debug logging if unneeded.

-- Almost always avoid updating a docker from within the docker, better to update the entire container.  Some programs store backups when updated, which will likely end up in docker.img.

 

As far as I know, I have mapped the downloads and so an correctly (tried to show you with the screenshots) out of the docker image to the appropriate paths on the share "data". 

 

Quote

9 hours ago, JonathanM said:

80% is about 16GB, not 737GB

When I hover over the 79% ( meanwhile it is already 80%) you can see that it is 737GB:

 

Quote

On 4/16/2024 at 3:46 AM, JonathanM said:

It's the percentage of space used INSIDE the docker.img file. Not the free space on the drive holding the image file.

 

I am not referring to the to the drive, but coincidentally it is 1TB. The docker.img is not 1TB, so how could it be INSIDE?

I hope you can see where my confusion comes from...

[JonathanM]

3 hours ago, stainless_steve said:

The docker.img is small as it should be according to the "file explorer" (don´t know the proper name):

According to your previous screenshot you aren't using that file.

 

[itimpi]

According to the diagnostics you have configured docker to not use an image file, but to go straight to the native file system for the container binaries.   You also have all of the ‘system’ share on disk1, so the size being reported for docker relates to disk1.   Normally one wants that share on the cache for maximum performance.

 

Finally you have the ‘appdata’ share partially on disk1 and partly on the cache but you also have it configured so that mover will take no action on files in that share.

[stainless_steve]

Thank you both for your answers.

I have built a completely passively cooled system with only SSDs.

As the first server I build was with a cache drive, but I had so many troubles that I could not solve, I started from scratch.

Initially I wanted to use that cache but then decided to don´t use it, as I only have SSDs and therefore no spinning up and down of hard drives. But that is the reason, I still have some data on cache and the array. I decided to leave is as it is as it was working for the time being and with all SSDs there should be no performance issue.

 

System is set to use all disks

 

appdata is set to only use cache

 

[stainless_steve]

@JonathanM oh so you mean, that the setting directory (did not change this, seems to be default) made it that nothing is stored in docker.img but in the folder "installed"?

But if so, shouldn´t be the maximum all that free space that is available on all included disks on the array instead of 1TB?

And can or should it be changed to a different setting? 

As I understand it, everything I do in a docker container would be stored in "appdata" like in my screenshot under "Default appdata storage location" as long as I don´t change the path mapping (which I have done).

[itimpi]

9 hours ago, stainless_steve said:

System is set to use all disks

 

Yes - but the way Unraid works the free space is the sum of the drives that actually have part of the share on it - not the disks that could potentially have it.  In this case the System share is only on disk1,

 

9 hours ago, stainless_steve said:

appdata is set to only use cache

 

It is set to use cache for all NEW files, but as there is no secondary storage set then you will see that it says mover will take no action, and thus existing files on the main array are left there.   If you want all the files to end up on the cache you need to temporarily set the array as secondary storage, run mover to get all files transferred to the cache and then only then remove the secondary storage settings.  You may need the docker stopped while doing this as mover will not move open files.  There are benefits to doing this in that once all of 'appdata' is on the cache it can be an Exclusive share which gives better performance.

 

8 hours ago, stainless_steve said:

h so you mean, that the setting directory (did not change this, seems to be default) made it that nothing is stored in docker.img but in the folder "installed"?

 

Actually setting docker to use a folder rather than a docker.img file is NOT the default so you must have changed it at some point.   With that setting the docker.img file is not being used.

[stainless_steve]

On 4/18/2024 at 12:18 AM, itimpi said:

 

Actually setting docker to use a folder rather than a docker.img file is NOT the default so you must have changed it at some point.   With that setting the docker.img file is not being used.

Thanks! Maybe I followed I guide and I can´t remember. Can I just change it to vxfs or vbtrfs? Would that be better?

 

All in all it comes down to 2 questions: Will i run into troubles if I reach the 100% and if so, what can I do to prevent this?

 

I am now at 90% and lost.

Can I change any settings that will expand the available space to more than 1TB? Or do I have to set my server from scratch again?

I am a noob and still wrapping my head around this.