February 13, 20251 yr Since upgrading to 7.0, my RAM is constantly full — out of 16GB, System eats up ~8-9 GB. Steps I've taken, to no avail: - Reboot (it starts off with 1.x GB, but then creeps back up to 8-9 GB) - Stop all Docker containers I'm attaching the diagnostics for reference. Hope someone can point me in the right direction. tower-diagnostics-20250213-0801.zip
February 13, 20251 yr Community Expert Immich appears to be the problem, disable the docker service, reboot in safe mode and see if the usage remains normal, if yes, start enabling the containers one by one and retest.
February 14, 20251 yr Author On 2/13/2025 at 10:14 AM, JorgeB said: Immich appears to be the problem, disable the docker service, reboot in safe mode and see if the usage remains normal, if yes, start enabling the containers one by one and retest. Can you tell me where/how you see this, i.e. which log file? Strangely enough, I stopped all containers and started them one by one, and everything remains normal. Then after a few hours, RAM is full again. So it seems to be a problem that only occurs / worsens over time.
February 14, 20251 yr Community Expert 24 minutes ago, keeny said: Can you tell me where/how you see this, i.e. which log file? Strangely enough, I stopped all containers and started them one by one, and everything remains normal. Then after a few hours, RAM is full again. So it seems to be a problem that only occurs / worsens over time. The OOM reaper is killing your immich process and also your VM - it's likely whatever immich is doing running the host OOM Feb 13 00:00:23 Tower kernel: Out of memory: Killed process 30380 (immich) total-vm:23196948kB, anon-rss:6242364kB, file-rss:52kB, shmem-rss:0kB, UID:99 pgtables:25348kB oom_score_adj:0 Feb 13 00:01:47 Tower kernel: Out of memory: Killed process 688879 (immich) total-vm:23163516kB, anon-rss:6202436kB, file-rss:4080kB, shmem-rss:0kB, UID:99 pgtables:23980kB oom_score_adj:0 Feb 13 04:30:47 Tower kernel: Out of memory: Killed process 12416 (qemu-system-x86) total-vm:6920960kB, anon-rss:2190524kB, file-rss:512kB, shmem-rss:7168kB, UID:0 pgtables:5576kB oom_score_adj:
February 16, 20251 yr Author I tried everything, but can' really pinpoint the issue. Immich doesn't seem to be the problem (I kept it OFF for a few days), and this morning, the same high RAM usage again. Attached another diagnostics report, and also ran df -h, which gives the following: Filesystem Size Used Avail Use% Mounted on rootfs 7.8G 7.1G 739M 91% / tmpfs 128M 1.7M 127M 2% /run /dev/sda1 2.0G 1.4G 681M 67% /boot overlay 7.8G 7.1G 739M 91% /usr overlay 7.8G 7.1G 739M 91% /lib tmpfs 128M 1.5M 127M 2% /var/log devtmpfs 8.0M 0 8.0M 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 1.0M 0 1.0M 0% /mnt/disks tmpfs 1.0M 0 1.0M 0% /mnt/remotes tmpfs 1.0M 0 1.0M 0% /mnt/addons tmpfs 1.0M 0 1.0M 0% /mnt/rootshare /dev/md1p1 2.8T 1.4T 1.4T 52% /mnt/disk1 /dev/md2p1 2.8T 1.4T 1.4T 50% /mnt/disk2 /dev/md3p1 2.8T 962G 1.8T 35% /mnt/disk3 /dev/sdf1 932G 48G 884G 6% /mnt/cache_ssd /dev/nvme0n1p1 932G 107G 825G 12% /mnt/vm_docker_nvme shfs 8.2T 3.7T 4.6T 45% /mnt/user0 shfs 8.2T 3.7T 4.6T 45% /mnt/user /dev/loop2 1.0G 7.4M 904M 1% /etc/libvirt overlay 932G 107G 825G 12% /var/lib/docker/overlay2/99104ffb6dc88362e7a994d4cef57a92249616fc8bbf102103fb914ea9b7c852/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/5f6857f58f69be250866e0cd806f1d6323f621d07a69af9237724833f0a7355a/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/21a31d956319606039b8bfab67af6c8027026bd8762d2f3ce2e0c27589348154/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/4a89c6a97f647c8cffc0247c395c968f5d0430fad41ab6c61a98612092476a81/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/6fcf67c53022a955bf58b9bd6e234c40861044acc28a8bc8efb9378ee43085a8/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/41800597878cde271469864fb48f903b20f02158d4e839fc955fdf44a844b519/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/6017850d4657e6758a6d160a3bccc3b04ca76450a37895f849c277bedff1f6a0/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/005f6210adb36796f75a28d971ea4d3d7c6ab19c48d80d72e68549d41e86ecc0/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/76475cff340045b3b7446cc151e97ccab4754e53ee420d03c7b688f564fe9b6b/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/2e6f478b3a8dda2306843b3cfa9d06f77af95076257fabf99f96097eeb300ddb/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/85b455ddd7ce479c36f40f169bfacab483f08bd808ec925881eb0beac28a15a0/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/47bac1ad4843d4b33d52be61cb404cdfc3576f5155f73efa63960a1cf7e5042f/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/7de7dbc3ab733eacde3e4393be88cea9ec0b049c1a36e7ec093623de16d6ec3a/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/1fa90ea935be2b802e346d742ed5e7b13d773cd735fccc472f64bb4c99b16422/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/15d757f861164c646801a4d4b20ba17818038ff2199fcdaccf1caa4fe2f61cdf/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/6b8d022acb9d0ee5e606a6d5758211f1614f211f9ebe8a4e0ddd72998cadc000/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/028beefa7bb95fb7f1c9f986038a042c9700e76ed2b7c011075dca3b05550210/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/d5ecaabf13f9e90b277e0956f029e020e46f78ce6fce2f43a01676c166bf0731/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/3d2c419ee802880f4281a1afcc71fa7585f01c1f9542d5042a6d601244394ae7/merged overlay 932G 107G 825G 12% /var/lib/docker/overlay2/8004b031e99eae6818200f36cd608acbe827d8843a098ff709ec4d0a05c1fd69/merged tower-diagnostics-20250216-0857.zip Edited February 16, 20251 yr by keeny
February 16, 20251 yr Community Expert 2 hours ago, keeny said: Filesystem Size Used Avail Use% Mounted on rootfs 7.8G 7.1G 739M 91% / This is a problem, you will need to find what is writing to RAM, possibly a bad mapping somewhere, like in a container or script, anything mapped outside a valid /mnt/user or /mnt/disk path will end up in RAM.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.