I have had this issue where my docker image fills up every few night and then it stops doing whatever is happening and it reverts back to the size before these excessive writes. I strongly believe it is plex based off the library actions that happen every few days. It always happens at the same time around 2AM (plex, as shown below.....).

 

I have roughly 20gb extra in my docker image. Any ideas or tests? I've looked at quite a few things over the last year or so and I just can't quite pin it down other than I have a very strong suspicion it is plex. I have tried looking at logs and I wasn't able to find anything indicative of this issue, but I may have been looking at the incorrect logs.

 

I am sorry to start another docker image filling up thread. I really have put in a lot of effort trying to resolve this.

 

 

There is nothing out of the ordinary in the sizes, IMO.

 

Name                              Container     Writable          Log
---------------------------------------------------------------------
deepstack                           4.28 GB       483 kB      1.19 MB
anaconda3                           3.98 GB       340 kB      52.2 kB
office-document-server-ee           3.35 GB       345 MB      3.60 MB
GitLab-CE                           2.46 GB          0 B      21.1 MB
MakeMKV-RDP                         2.45 GB      22.8 MB       188 kB
digikam                             1.77 GB       625 MB       140 kB
Collabora                           1.54 GB      36.4 kB       199 kB
deepstack-ui                        1.40 GB       289 kB      1.93 kB
binhex-readarr                      1.29 GB       135 MB      8.28 MB
Krusader                            1.26 GB      32.5 MB      96.8 kB
HandBrake                           1.15 GB      99.0 MB       172 kB
binhex-delugevpn                    1.14 GB      3.14 MB      37.5 kB
double-take                          995 MB         55 B      2.99 kB
Nginx-Proxy-Manager-Official         905 MB      13.9 kB      7.88 kB
macinabox                            899 MB      3.43 MB      39.7 kB
Rocket.Chat                          875 MB          0 B      47.2 kB
openhab                              847 MB       149 MB      3.72 kB
chia                                 745 MB      1.62 MB      5.13 kB
Sonarr                               738 MB      97.5 MB       855 kB
luckyBackup                          719 MB      2.04 MB      8.92 kB
plex                                 713 MB      64.1 MB      1.13 MB
MongoDB                              698 MB         20 B      6.65 MB
binhex-syncthing                     693 MB      1.81 MB      17.0 kB
code-server                          630 MB       316 kB      25.8 kB
Ghostfolio                           565 MB      1.34 kB      64.1 kB
NodeRed                              535 MB          0 B      13.2 MB
matrix                               484 MB      29.7 kB       102 kB
nextcloud                            444 MB      35.0 kB      36.0 kB
lidarr                               406 MB       140 MB      9.23 MB
postgresql14                         376 MB         63 B      1.54 MB
Influxdb                             308 MB      2.83 kB      1.57 GB
mariadb                              293 MB      16.5 kB      55.6 kB
FileZilla                            215 MB          0 B      21.1 MB
Grafana                              206 MB          0 B       767 MB
Beets                                200 MB          0 B      21.1 MB
bitwardenrs                          193 MB          0 B      3.73 MB
Headphones                           192 MB      22.1 MB      5.48 MB
radarr                               190 MB      19.5 kB       263 kB
flame                                179 MB          0 B      3.31 kB
sabnzbd                              177 MB      20.3 kB      4.58 MB
redis                                164 MB      68.4 MB      20.2 MB
heimdall                             127 MB      41.7 MB      4.89 kB
netbootxyz                          84.7 MB      11.1 kB      6.35 kB
element-web                         69.2 MB        769 B      8.70 MB
Authelia                            50.0 MB        190 B      7.37 kB
Cloudflare-DDNS                     36.3 MB      5.81 kB      3.16 MB
synapse-admin                       29.3 MB      1.09 kB      54.4 kB
MQTT                                25.7 MB          0 B      1.24 MB
---------------------------------------------------------------------
Total size                          41.1 GB      1.86 GB      2.50 GB
 

 

Edited March 15, 20233 yr by live4soccer7

If you think it is plex then you should post the settings you are using for the Plex container. 

Not a problem. Here we are:

 

 

I think that covers what is relevant. There are a few more ports below the last screen shot, but I can't see how it would be a part of the issue.

I don't use plex, but I think if you don't assign an external folder for transcoding, it happens inside the image.

That could be the case and something I will look at, but it wouldn't explain why the docker image fills up around 2AM every few days. Plex is being watched here throughout the day and we are sleeping at 2AM. That coinciding with the "scheduled task" within plex itself.

Do you do any transcoding?    You do not have that mapped to an external location so any work files created by transcoding would be written internally to the docker iimage.     In addition any temporary files written to /tmp inside the plex container would also be written within the docker image.

Would it do transcoding while "processing the library" on a scheduled task? I would assume to just transcode to a location on the cache disk would be best or should I try to push that to ram? Whatever it is, it is consuming at least 20GB during this time, for whatever reason

 

It happened last night. IMO, it has to be a local process to fill up 20GB that fast. My internet connection is not that fast for any kind of downloading. I added a transcode path to the /mnt/cache/transcode_plex and then set the transcode within the actual plex docker to /transcode

 

Doing that, it can't be plex transcoding or processing. Is there an obscure log I can check to see what is doing all the writing at this time? I mean, it fills 20G roughly in 8 minutes and then clears itself.

 

Edited March 16, 20233 yr by live4soccer7

I remember having a similar issue. For me if I restarted the docker the size returned to normal. It turned out to be temp files generating inside the docker image. I added the following temp mapping and it went away. 

9 minutes ago, Gragorg said:

I remember having a similar issue. For me if I restarted the docker the size returned to normal. It turned out to be temp files generating inside the docker image. I added the following temp mapping and it went away. 

 Thanks, I'm giving it a shot. I haven't be up at 2AM when it happens so checking the docker size to actually see the culprit hasn't been much of an option. Now that I'm documenting it a little here if it continues to happen on a set schedule then I'll stay up or wake up a 2AM and check dockers sizes to see which one it is. It will at least give me more direction.

No issues today. I still believe it is every few days roughly. I'll post back up tomorrow morning.

Still nothing. Fingers crossed. There are several hundred jpegs in the tmp folder that was created, as suggested by @Gragorg

Edited March 18, 20233 yr by live4soccer7

Still no filling up of the docker.img. I'm thinking /tmp may have resolved the issue. I'll let it go for the next week before marking this as resolved. This has been bugging the heck out of me for a long time. It may be a little premature, but thank you everyone for helping me save my sanity.

Still no filling up of the docker.img. I'm thinking /tmp may have resolved the issue. I'll let it go for the next week before marking this as resolved. This has been bugging the heck out of me for a long time. It may be a little premature, but thank you everyone for helping me save my sanity.

Still looking good!

Noooooooooooooooo! lol. At least I know it is on a 7 day cycle. I posted up the previous screenshot on the 16th.

 

It has to be this, but why would it be writing to the docker.img

 

It is definitely the "optimize database". I ran it manually and 10 minutes later the docker image started filling up. Again, I have 20GB approximately of headroom within the docker.img. I'm using the linuxserver plex docker.

 

With this, anyone have any ideas/suggestions?

Mar 24, 2023 12:30:10.350 [0x14fef3c16b00] ERROR - [Req#11ea90/Database optimization/com.plexapp.plugins.library.db] SQLITE3:0x80000001, 13, statement aborts at 26: [INSERT INTO vacuum_db.'statistics_media' SELECT*FROM"main".'statistics_media'] database or disk is full
Mar 24, 2023 12:30:13.997 [0x14fef3c16b00] ERROR - [Req#11ea90/Database optimization/com.plexapp.plugins.library.db] SQLITE3:0x80000001, 13, statement aborts at 1: [vacuum] database or disk is full
Mar 24, 2023 12:30:14.015 [0x14fef3c16b00] ERROR - Thread: Uncaught exception running async task which was spawned by thread 0x14fef3004b00: sqlite3_statement_backend::loadOne: database or disk is full
Mar 24, 2023 12:30:14.023 [0x14ff03e92b00] ERROR - [EventSourceClient/pubsub/172.105.245.178:443] Retrying in 15 seconds.
Mar 24, 2023 12:30:14.024 [0x14fef401cb00] WARN - [Req#11eb05/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.030 [0x14fef3810b00] WARN - [Req#11eb0c/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.038 [0x14fef3207b00] WARN - [Req#11eb13/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.047 [0x14fef3004b00] WARN - [Req#11eb23/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.048 [0x14fef5034b00] WARN - Held transaction for too long (/data/jenkins/server/3535474905/Statistics/StatisticsManager.cpp:252): 0.250000 seconds
Mar 24, 2023 12:30:14.048 [0x14fef5c46b00] WARN - Took too long (0.190000 seconds) to start a transaction on /data/jenkins/server/3535474905/Versions/VersionGarbageCollector.cpp:89
Mar 24, 2023 12:30:14.048 [0x14fef5c46b00] WARN - Transaction that was running was started on thread 0x14fef5034b00 at /data/jenkins/server/3535474905/Statistics/StatisticsManager.cpp:252
Mar 24, 2023 12:30:14.054 [0x14fef2bfeb00] WARN - [Req#11eb2e/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.064 [0x14fef27f8b00] WARN - [Req#11eb38/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.066 [0x14fef5034b00] WARN - Held transaction for too long (/data/jenkins/server/3535474905/Statistics/StatisticsManager.cpp:255): 0.120000 seconds
Mar 24, 2023 12:30:14.068 [0x14fef25f5b00] WARN - [Req#11eb42/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.076 [0x14fef21efb00] WARN - [Req#11eb4e/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.085 [0x14fef1de9b00] WARN - [Req#11eb59/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.093 [0x14fef19e3b00] WARN - [Req#11eb62/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.106 [0x14fef11d7b00] WARN - [Req#11eb6a/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.107 [0x14fef17e0b00] WARN - [Req#11eb71/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.114 [0x14fef17e0b00] WARN - [Req#11eb79/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.121 [0x14fef09cbb00] WARN - [Req#11eb82/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.122 [0x14fef07c8b00] ERROR - Caught exception trying to stream file: /usr/lib/plexmediaserver/Resources/Plug-ins-914ddd2b3/WebClient.bundle/Contents/Resources/js/chunk-81-a3acf44dc3e4f508e0a8-plex-4.87.2.25887-d04a1ad.js: write: Connection reset by peer
Mar 24, 2023 12:30:14.123 [0x14fef07c8b00] ERROR - Caught exception trying to stream file: /usr/lib/plexmediaserver/Resources/Plug-ins-914ddd2b3/WebClient.bundle/Contents/Resources/js/chunk-272-b354b4fcac6e0b461c38-plex-4.87.2.25887-d04a1ad.js: write: Connection reset by peer
Mar 24, 2023 12:30:14.123 [0x14fef04c5b00] ERROR - Caught exception trying to stream file: /usr/lib/plexmediaserver/Resources/Plug-ins-914ddd2b3/WebClient.bundle/Contents/Resources/chunk-328-bc73675043bc14e1e270-plex-4.87.2.25887-d04a1ad.css: write: Connection reset by peer
Mar 24, 2023 12:30:14.124 [0x14fef09cbb00] ERROR - Caught exception trying to stream file: /usr/lib/plexmediaserver/Resources/Plug-ins-914ddd2b3/WebClient.bundle/Contents/Resources/chunk-272-b354b4fcac6e0b461c38-plex-4.87.2.25887-d04a1ad.css: write: Connection reset by peer
Mar 24, 2023 12:30:14.124 [0x14fef0dd1b00] ERROR - Caught exception trying to stream file: /usr/lib/plexmediaserver/Resources/Plug-ins-914ddd2b3/WebClient.bundle/Contents/Resources/js/chunk-328-bc73675043bc14e1e270-plex-4.87.2.25887-d04a1ad.js: write: Connection reset by peer
Mar 24, 2023 12:30:14.127 [0x14fef09cbb00] WARN - [Req#11eb92/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.134 [0x14fef00bfb00] WARN - [Req#11eb9d/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.138 [0x14feefebcb00] WARN - [Req#11eba3/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.145 [0x14feefab6b00] WARN - [Req#11ebac/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.150 [0x14fef00bfb00] WARN - [Req#11ebb6/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.156 [0x14feef8b3b00] WARN - [Req#11ebbf/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.163 [0x14feef4adb00] WARN - [Req#11ebc9/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.168 [0x14feeeca1b00] WARN - [Req#11ebcf/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.175 [0x14feee89bb00] WARN - [Req#11ebd9/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.180 [0x14feeea9eb00] WARN - [Req#11ebe1/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.187 [0x14feee292b00] WARN - [Req#11ebec/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.191 [0x14feee08fb00] WARN - [Req#11ebf2/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:14.468 [0x14fef4a2bb00] WARN - MyPlex: attempted a reachability check but we're not yet online.
Mar 24, 2023 12:30:15.091 [0x14fef401cb00] WARN - [Req#11ebfa/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:15.097 [0x14fef4e31b00] WARN - [Req#11ec02/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:15.103 [0x14fef360db00] WARN - [Req#11ec06/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:15.109 [0x14fef3810b00] WARN - [Req#11ec0b/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:16.126 [0x14fef0bceb00] WARN - [Req#11ec54/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:30:30.600 [0x14ff04095b00] ERROR - [EventSourceClient/pubsub/172.105.245.178:443] getaddrinfo(172-17-0-1.abcdefghijklmnopqrstuvwxyz012345.plex.direct) failed: -2
Mar 24, 2023 12:30:46.122 [0x14fef3207b00] WARN - [Req#11eef4/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:31:16.139 [0x14fef3207b00] WARN - [Req#11ef32/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:31:46.132 [0x14fef3207b00] WARN - [Req#11ef65/Transcode] Got a transcode session ping without a valid session ID.
Mar 24, 2023 12:32:16.149 [0x14fef563db00] WARN - [Req#11efa8/Transcode] Got a transcode session ping without a valid session ID.

 

 

This is the only thing in the logs I could find that were during the optimization time.

You probably need to ask the actual Plex developers what their application does when "optimizing the DB" and what additional directories you may have to map so it doesn't write inside the Docker image.

Is there a log that I can take a look at to see what it is doing at this particular time?

I'm still at a loss on this and I have gone to the other forums and there has been no reply with a couple bumps.

 

Any ideas at all?

I didn't really solve this issue, but it is plex doing it. I bumped up the docker image another 30G or so and plex does whatever processing it is doing and it reverts back to normal. At least it isn't completely filling up the docker image and potentially causing issues. 

Past few days I've suddenly seen my docker image filling up and found the culprit to also be plex. I have /transcode mapped to /tmp and also a PlexRamScratch ram disk setup. 
Sounds similar to what you're seeing. If I find the root cause will update here as well. 

I just upped the docker img size.