[Solved] Docker Image Usage high


Recommended Posts

Hi, 

 

I've been trying to track down my docker image usage. 
I haven't found much in the way of rhyme/reason on how it's getting larger.  Seems to be about 3% at a time though.

 

  • System Version: 6.9.1
  • System Memory Usage:
    image.png
  • Docker Information:
    image.png.8bfafb102dd685203ee2f84aed33e46a.png
  • Docker Settings:
    image.png.07f851e01ba749eef06eb642bce9ea77.png
  • All docker containers have a "/" prefix before their path
  • Log rotation is enabled
  • Container size output:
    image.png.f8baec12b2890a1f896a415f08ea1a83.png
  • Output of command
    du -ah /var/lib/docker/containers/ | grep -v "/$" | sort -rh | head -60
    image.png 
  •  Output of command
    find /var/lib/docker/containers/ -type f -exec du -Sh {} + | sort -rh | head -n 15

    image.thumb.png.fdb8dd255f14871b7ea09b1722b93587.png

  •  Output of command
    docker system df -v
    image.thumb.png.723cd09697f845631c50efa38c3de6c9.png
  • I've been using this to find the largest folders on some containers
    du -Sh | sort -rh | head -5

     

  • I've been using this to find the largest files on some containers 
    find -type f -exec du -Sh {} + | sort -rh | head -n 5

     

 

Link to comment
9 hours ago, John_M said:

You probably have a bad mapping somewhere and one of the containers is storing files internally instead of externally. You could try looking at the docker run command for each container.

 

I've gone through all my mappings and they seem correct.  They all are defined based on the template, docker run shows the prefixed "/"

 

Output of Command:

du -h -d 1 /var/lib/docker/

image.png.e8033a20eef737bafc80cb21bf7af85d.png

 

I've also been running the following command to see file size changes

docker ps | awk '{print $1}' | grep -v CONTAINER | xargs -n 1 -I {} sh -c "echo -e \\\n#================\\\nDocker container:{}; docker exec {} du -hx -d1 / "

 

Link to comment
59 minutes ago, ShadeZeRO said:

 

I've gone through all my mappings and they seem correct. 

Have you matched what's in the mappings to the internal configuration of each app? If the app is set to write to /Downloads and the mapping is /downloads you will end up writing to /Downloads inside the docker image.

Link to comment
Posted (edited)
22 minutes ago, jonathanm said:

Have you matched what's in the mappings to the internal configuration of each app? If the app is set to write to /Downloads and the mapping is /downloads you will end up writing to /Downloads inside the docker image.


NZBGet is the only application that downloads anything.  It's been working fine over over a year.
Anytime I select a path, I choose it through the drop down so anything case-sensitive is preserved. 
image.thumb.png.8141d39b711f53ca5aa906b654b07c8d.png

image.thumb.png.ce3c0b43eadd4b54a7a5e058833f8a01.png


Reslio is another App that pulls data in occasionally:
image.png.0f262251c75d55bfe66069f2b2bda935.pngimage.thumb.png.4052e789dbffa580089d6a2d9ef7375d.png

 

Plex (mostly default settings)
image.thumb.png.54ccbddd33f6963420c9fe7394f2081a.png

image.png.9ea38ce2624f9789e3daf27abe51dc83.png


Splunk:

image.png.53000c24bae6c57c543f530d9bd3d4f2.png

image.png.2bde167831c0febe771a16ee35616a84.png

 

Edited by ShadeZeRO
Link to comment
22 hours ago, John_M said:

You could try narrowing down the problem by running only some of your containers for a while. With the various tools and charts you've shown you should be able to eliminate several of them quite quickly.


I did some considerable digging.  It ended up being the "splunk" container.  
This was determined by doing a "force upgrade" and seeing the behavior after a restart.  The docker image usage spiked every time that action was performed.

 

After that, I shutdown docker, deleted the vdisk, and recreated (as XFS since this was a good opportunity to move off BTRFS) and re-imported my templates (except Splunk).

The usage has been exemplary now.  

 

Thank you to everyone who offered support.

Link to comment
  • ChatNoir changed the title to [Solved] Docker Image Usage high

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.