Docker high image disk utilization: why is my docker image/disk getting full?


Recommended Posts

  • 3 months later...

I finally solved this for my setup today and wanted to share what worked.

I had a situation where I had orphaned volumes. 

I figured it out by using commands from earlier in this thread.

I ran this command from earlier in the thread from beastieg on 2/4/2021

docker ps | awk '{print $1}' | grep -v CONTAINER | xargs -n 1 -I {} sh -c "echo -e \\\n#================\\\nDocker container:{}; docker exec {} du -hx -d1 / "

It showed that none of my active containers were using much space.  

I then found this stackoverflow post on deleting orphaned volumes

https://stackoverflow.com/questions/27812807/orphaned-docker-mounted-host-volumes

 

I ran this command that showed I had 3 orphaned volumes:

docker volume ls -qf dangling=true

 

I then ran this one to delete them:

docker volume rm $(docker volume ls -qf dangling=true)

 

I hope if you ended up here as I did after googling the high utilization message that unraid presented, that this might help you as well.

Good Luck.

Link to comment
  • 5 weeks later...
On 10/10/2022 at 8:47 PM, andy10102022 said:

I finally solved this for my setup today and wanted to share what worked.

I had a situation where I had orphaned volumes. 

I figured it out by using commands from earlier in this thread.

I ran this command from earlier in the thread from beastieg on 2/4/2021

docker ps | awk '{print $1}' | grep -v CONTAINER | xargs -n 1 -I {} sh -c "echo -e \\\n#================\\\nDocker container:{}; docker exec {} du -hx -d1 / "

It showed that none of my active containers were using much space.  

I then found this stackoverflow post on deleting orphaned volumes

https://stackoverflow.com/questions/27812807/orphaned-docker-mounted-host-volumes

 

I ran this command that showed I had 3 orphaned volumes:

docker volume ls -qf dangling=true

 

I then ran this one to delete them:

docker volume rm $(docker volume ls -qf dangling=true)

 

I hope if you ended up here as I did after googling the high utilization message that unraid presented, that this might help you as well.

Good Luck.

I had the same issue with orphaned images. Someone mentioned it earlier in this thread and so I turned on "Advanced View" while viewing Dockers and all the way at the bottom it lists them all. I started to delete them one by one and thought there must be a better way to get rid of these all. 

 

There is. If you have the user scripts plugin there is a script called "delete_dangling_images." I ran that and it cleared them all. Went from 75% docker image utilization to 37%. The user script also allows you to set it on a schedule to automatically do this cleanup at a set interval. I have it set now to cleanup each month. 

Link to comment
  • 1 month later...
On 10/10/2022 at 8:47 PM, andy10102022 said:

I finally solved this for my setup today and wanted to share what worked.

I had a situation where I had orphaned volumes. 

I figured it out by using commands from earlier in this thread.

I ran this command from earlier in the thread from beastieg on 2/4/2021

docker ps | awk '{print $1}' | grep -v CONTAINER | xargs -n 1 -I {} sh -c "echo -e \\\n#================\\\nDocker container:{}; docker exec {} du -hx -d1 / "

It showed that none of my active containers were using much space.  

I then found this stackoverflow post on deleting orphaned volumes

https://stackoverflow.com/questions/27812807/orphaned-docker-mounted-host-volumes

 

I ran this command that showed I had 3 orphaned volumes:

docker volume ls -qf dangling=true

 

I then ran this one to delete them:

docker volume rm $(docker volume ls -qf dangling=true)

 

I hope if you ended up here as I did after googling the high utilization message that unraid presented, that this might help you as well.

Good Luck.

I just ran this command

docker volume ls -qf dangling=true

 

And the output for me is massive

0bea2853856b2ec930117b708ee704917be51eabe7c943b4a044d34555eac24a
0c9333424a9ef5fd4f9665c4d3cbe3e934d8b3c482409267b05b08119ed4b66f
0db05fd3e0ce3cffd2d8432e4678f9c7a1f6f67e21be8bdd6086c3068c3672d5
0e89f77bc341b134f18793828a287e2e05e23ce53ece4c3ed54675fc0b346ce2
1a0db80561a0372348444b9862590726f072e1bee592d8727b6f938c84976a41
1c2295838c14133e965cb3cee462922d80340168f5a33f1362a049a41ffa27c0
2b620234ae90eb5c7f5529c19a056a67a421897fc0d988988cfecdf54068193f
2be04af7ef26a2d7e079051efaa3b87f70cf83050fb27a7977cd9b0da38d738b
2d71af315c2b79ed0e52369425f481d6593f97f81f9029cef61e0fad3ffac2e4
2dae54ecd275a757af7f4edcb48f39b548493e2e844a74b377b4ffa9166869c6
2e9f787b5b84188c94e5d86297d93d3f6ecccc79d2cdc9a4d7a520ca63712255
2f1d1b060ea75c86bab906eedadbfb0a0591f8abb53d8a0a13f6fc3c260f3ad0
2f2117445f4d5d57e901cb4bb6f8296c062c02f6cf62c84240dd9a0a071c7644
2fb2eec397a6fc1ad66f72ef6b6bb620238f8405124f43b1ffc362b7ff6f7011
3b9118732e169bf25fb625d9a513f0a89b00b4b6fe6d526c59f7b671d03c6b27
5deaca2e3b19c895b4923478ae1385742fab5c6c8fca15d85b4f8126bc024980
5e24b558d161afaa2db51eaf43418690d462a41a259d9993a4f59eac8474515c
5fa34128a35546063252182c7e9d398d19b1eefe026a80bc925d160b575667db
6a38d95c43959eda90bdf0c53813d9ea1995503bfc0e2d0bfce1ce936cd3d532
6a2094af0c9d5624b9abd50ce4140a30990a27a7e23e38bcc02343916ba84cc0
6b8b7daa0446d2107f9e30d5d79fa0ec0f907022e63cb5c822eb5829e8d9382b
6f1c80a2d18122fd5ea5dd03d07c220384e8eabf77e8be6f0f371e62a1fc0f99
7a7d16050f1596a7e44f78bf08e462e6e1b3b793542360dd57e9e519e619319d
7abe60dd5e0b80190d4560615ce56cb13f05dd77e8eb56f1a112f68b8ddd1e7d
7b072bf0903a38cefcfde76938b5cbc3ee130904894286ab92929a7049bb9285
7bb97ec5b34f40f97104a094d497a50bcacc3b82219e224804a893278b6bd009
8e861df1c6e6e913633c895be89440ced5701e946f1090d9647ae18cc7e6cd35
8fd248cdc05467d310f315a7ef7bcf4fa52d60484a3eab9872997d64ecaf51c1
9a2cdb25fffa326defb64d874775a928a1298356ac7e270fb7f7822511d1e2e2
9c798273c00a193a519b03b1cee002233443d68222b184f6be5bc4662f10df53
9f81770cc9f21c7bebfd564fca26f94bcbf7cfff23d61a6324fd48ff7ac0ca02
10a4173a8db3e18760df3b77196daaee2d3d1c6616ff8b673da8196f21a1dea5
10f2bfb92d745fab1001bcd2e37919918311c86aa3fb387fc9f21e2f3e7cae82
14a104f37d821d0ec2b98285478ef0ba1fb6a0ef8207ffb93680a85ef1f22064
15c24646cfcbcd15112c8213d327be4f70d463da994aaf5ac932e4fc825108f0
15ffa279e66c0fc48e65d1ebdee9f67854c08af518e01614e4fdb5bf6c06b8f1
15ffd4eddc9e71a9c406df44eed130936cca1a4397763d4e39969dcd953acc82
18c80beb936f711b0616b1feebd0b97a3f907e64343245bfbdb4dc8cf73dd238
20aa5b4740b9d088debbf883689d94fa015e9f12c183df3b8303a6472b11c0d4
20ccc4ab1f07e9b8cee9cadf036bbb0a2d3dc94a81fafaeada8ea7cc764a49a2
34ed565915212d95ab30dad6c5c25b5d9230b9c3129abe32c9fd875e02605214
040c003858b3a51fb38b45397710f0f3ea7be9561b5c7dc230f77fa59094c15e
43b910d6273cf48f295e143bac2bdb2125f6347fa07e7686ba780d34d3af1fb5
49e8ba3d14d4f2396cd230e832d6b65ed42fd9a699f6f42a1743c7c86e56df8b
50cca50176b8e6457a75a0931c2b2f12cb669f6a7351f63f4ac775b4ca661acd
53bcca8e24bce1f99841af53be79dc8ab6cd2928d8b157dfbf0e3fd5de7ee1f3
70d9a08cf7471ebaf2e329239d01d9f9ac256acfd2a1f264bd5458214d06cab5
71dfc6b76a6290726dccab9f25a1890e5ba3e13bbc39f41ad934189bfc9f1ae1
072c1cb859f745bfb7dac3d6d405c8f985b19b48894952c4d5d13e7c57224d08
81c2c024e8b69bd017825b50f3b2edfd14cee5098985e98850459ddd5de64070
82b7bea867914851426728567e0a9e6d93271cc5f67d3f6f039583aafa773939
88a40147d5f9b0b5683864bdc11f38304cc659c6894abc430e82c74c1b29d48e
96d46d3461cc148afc730651019d1267ba0d8160b160579709690c3783f9c173
98b65b4ea820d3704a31687dffc3cdf489c4e8f306e772d57b1028a449191a83
168c6359fdf10295730f4cf589065e743bd5d9c14d1bb9670592f426f0481558
184d5ea782adc3199291a22ad548ab71ba9c45d68582a98a8a4bb6eea16b8d09
194d9a07d2597272d09fd18b71e1a8859db20fc56a17acb534e173b63677345c
202d4b6712824ae3d2a57d753c86ac0cd694e75a15b7f83919f0ef8072f498d1
240d71ba9ebf99f11650350cd94877a70d26cf002de0fc250800605673e97f32
412e49f17734508943e62a416d556ce815b594c3df1149133e4c1853ee8d2cdf
444cf49052c021d32aa7f3bb8d8cde3b06c3352e87382d493a47a376b7990eea
612ba4bd716a39bd371a44976f85a5ce4e8f82780a51c2970ff66e0572823b2d
720ce496b01188bf3add36272f0f21529740840179a4d8c29df862ab116082db
866d227a13282882270ea00703483c5944378660166ed0bef4af04bdf48327da
927e4ea73fe678c889329f8d9f9a287c8de758813436acfdf0f58176e2aefd69
986bf648f08113d37b45e15673518f90f77f65b0bf289af534a1d5f9db7e9b64
1589e09f22675b15e20096c0e34fa65514bed006be762835c55250c9f1aa5c1f
2442e0d951c1e91529b33ea26b900189493886bd5cfe392dedda7b0b4c96ffa3
7427b62d979235e2d8836d5b7e267f5c7273bd58050c5c4b0add5c69b5829fcc
11311d201d11afdce7d17918a9951e070fc67163ac02a9052d4d5a531ef5dddb
025731cce6d3268213b7c1d4f927c748b06f7fc1e32e6a6c8c06b4bfa4df0862
064867e809141fd69802d1fef33cf71b225bf98b4a6d1637b587603b2d402190
99309fb395190caab6c7d6d54c512cf1a53cb7b33d814fc637c64e01d6918bbe
350418f392905207793cbf17ace4e3874c7517bfb310b25cc456dbc96e7d1e59
669169cd8b54bfbbbeba5a4fab019a509c533c3131cfef492168b8bc3033e647
34091079f325c2978f25ab53385435c261b24d87482e03c940ce4c1789799f71
923772436c5b148be97832e75811de5cdec50e5ca038c9c542548a109b3d025f
a0d95d190d8e861d45df910eb26c2b1828da0914f302163e1bce1fce292117f1
a15c2ea51ce6516f9f3fd3ebb248d80182c9fdcdf4e03e94cbb3d4739299e7ff
a26a8ed3f4c20f5567bc9342ec15f81725af1aec40cc2be67253c9440c2fed02
a431d80aa0e3e137688905816b15a697262c93835859ec6a925b66406abaf864
ac04168a0e22f8875802c96ce3c2e3c7c6f8651294c0c78880fe6257afe6f3a1
ae76f0c7d849490a01d025fa88eebd92620f8d12642d2ebb61ecc8c49ee81843
b1e8a22b377a89cf8b6ce472fb65f1cc96a0d938d0d6a01db1da40ff3fdbbfd0
b7379a35c9b5b21d35036ad542b39b2a469228528f5d987065fcc0ed2038dd17
b7468c233eefebabd9860b511e03f2cfaabd99ff4926878d8aa8f1ddb9f7e54c
bb3dc594affb2116bb412cf395b8682aa9bcc1517a93bb325d8efa3790f6ee1d
bb55f7aa7e33b1643446f981e66f847f64968bfc184d04800d513d065aabf65c
bd76464c210211db45e51bcdceacd7f0c4ed942157d0bad21094727f6519fa4e
c1d8a3903fa4812e95d04a420aeb177ab900dd652e2cffe3b622064605e73d46
c1e4c1ffcad2e3d98d1e155b18964b7ec9d2287c9be84b589a54a15a5f79f91b
c6a05e0608bd6f95df123efc5c6da1467493927eb36bcde4321823f29eb74567
c169f168bab7f4e769123d6cf891d6def927bab894f5489c0ed0292daeea45e3
cdfb3098b0a6ba1d1e1ee19e48347d910bcdd83d091c9d803889f7fb68e562f4
ce4350df6c997e08ea4dc37309ad8c0cb2cf59d96b7e0eec807a3490647c69b5
d8b318c6b7bf0189e70a43fce9ac6eb0e22ad162c3ae6cc8cc7f681e011ceb61
d19e720e5de5dc343d31b2365157dacdc7e21159c173870fe15f79997db1052c
d25fa6b1a8bd293d30d61d34f52565efc2174efec74ba641d3b264618bc955e0
d68f5a47b0615ec55456c9b6e1baf3dd6019aaa8cbb8a17b3a22f73f68aa8575
d562e1906b381d25619db7a491df5c16727e1ba34e290972932d0b87d4cd1fee
ddbd79c49eece49e121ef1c94cac53d471ff349a74aaf9bb69710e6bca5e1dbb
e4fa8198fa4cb6c883c348e3faac6c1ebeac5a65fa9def8796277550fe925592
e8b1e42fc6b3d6ec8b18a106d987eff595def53d592c31798987fbf17f1d3a39
e451bf88389e3788ea843dc2c6f5387e458f23ed1588779656c26d24b992b5b8
e6236b1bdc2f913311e912117a0571e458b963dba25988ad4970a17a35fe3167
ed7b1124c731639b7f59e351f6ed4ddf2dcfe777350b5136875e78760b4fa899
ed591d6cd15c1814763e1c5c198e11acdc09e73883a32af7786e74548e417dc7
f07df1b5d3ea8b7810dad3d61884254eaba5acf8d7cedf31631be89b3a37df42
f74eee759c004e4e9a6806318dfe5b760e6f92189f053e616fe72c736a962bb5
fcb3863abaa130c1042bd4668e3b5a8f7715bfcc56dda830488804ad0a97f45d
ff2a6d06071b500409c03d9b7e99b974411fc5c00c3bcbed0eb944a18416edc7

 

I do have 54 container, but are these all orphaned volume? I don't see anything orphaned on the docker tab.

Link to comment
  • 3 months later...

I’ve tried everything listed here. Calculating the docker image size takes forever then spikes all cpus and I need to

stop and start docker to get back to normal. Running the script from @SpaceInvaderOne does the same thing and outputs nothing. Had to put my img file to 100g just so I can make it 2 weeks without deleting my image. Never had an issue for years until a failed Plex docker update where it would not delete the old image. I had to stop docker, delete the image and reinstall. Now I fill up about every 2 weeks and need to delete the image and start over. Once everything is installed I run about 6% for a week. Friday I was 40% sunday 70% yesterday 80% today 88%. Not sure what to look at next. 
overnight around 2am I get 3-4 emails in a row as the % goes up by about 2% with each email. I suspect some sort of nightly backup like Plex. But all the mappings seem ok.  

Edited by macbentosh
Link to comment
  • 3 months later...

ich greife das hier mal auf, weil ich nicht weiter weiß...

 

Vorab: meine Docker liegen im Verzeichnis nicht in einer Image Datei.

 

Ich habe insbesondere das verlinkte Video angesehen und den Lösungsansatz über das Script etc. ohne Erfolg versucht. Meine Containergröße ist bei entspannten 3,34 GB aber in der Übersicht belegt mein Docker 89% / 14,5 TiB. Ich habe nur 3 Docker und ich denke die sind ordentlich konfiguriert. Ich habe auch alle Docker gelöscht und neu „installiert“ also runterladen lassen. Kann es sein, dass meine komplette Festplatte als Verfügbarkeit angesetzt wird und es gar nicht an den Dockern liegt, sondern der Platz durch meine normalen Daten „gefressen“ wird und die Anzeige daher korrekt und „normal“ ist?

Link to comment
  • 4 weeks later...
  • 1 month later...

Hi there. I am trying to create my second backup station. Resilio sync is main docker app (and right now only one) to use. In first station everything is going well (using much more docker apps there), but with this one I got into trouble with 100% full docker (20 GB). Temporary I increased space to 30 GB, but need to fix what the problem is. I am using 2 discs each 10 TB (one parity, one regular disk of course). Disks are 10% full right now. I am planing to use one SSD as cache, but in first station, which works well I have none cache disk (just five 6 TB disks). Can someone check diagnostic file to lead me where the problem came from? Thx so much in advance.

hometower-diagnostics-20231001-1114.zip

Link to comment

I found that resilio "log" part is so hudge....so I added "--log-opt max-size=50m" parameter to docker and checking what's going on. Is it right way? Anyway, I do not remember anything like this to be doing on my work Unraid (first one). In ResilioSync power settings there is log value as "100". Do not know what that mean, but will go to check work Unraid what value is there.

Link to comment
  • 5 months later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.