Michael_P

Looks like on the 2nd and 18th of June Plex media scanner hit its memory limit and was killed. You can check the container logs to see if there's a file it doesn't like.

Reboot to clear the log so FCP stops warning you

Back on APR 12 one of your containers (8c565a95c675) hit its assigned limit and was killed. Looks like a one-off occurrence so you can reboot to clear the log so FCP stops warning you. If it happens again, you can check the container's config or assign it more memory

Looks like back on the 23 March whatever container matches this ID 7f7c77b7b0989 ran over its assigned limit and was repeatedly killed, then on the 24th it looks like you were running something that had ffmpeg doing work which tipped the host OOM so it stopped the highest consumer which at the time was your VM.

Doesn't look like you need to do anything, so just reboot to clear the log and FCP will stop warning you

Yeah, stress ran the host OOM, reboot to clear the log so FCP stops warning you

With 16G of RAM you shouldn't be using it as cache, it's actually recommended to use an old SSD for cache as the file sizes can be pretty large if you're working with blurays and such. Map the cache location to a disk.

Looks like one of your services tipped the host over and it killed the VM back on 5th and again on the 8th. If it happens again, you will need to add more RAM or put your VM on a diet if you plan on running other services like that regularly.

Reboot to clear the log so FCP stops warning you

Looks like yesterday twice at around 1100 and 1145 duplicati tripped the host out of available memory - if you were doing something with it, there's your problem. It's not related to bad RAM, if that was your concern. If you've already rebooted to clear the log, FCP should stop warning you, and if it happens again you can configure the container so it's not storing so much in RAM.

Looks like on the 15th of Nov and on the 14th of Dec a python process ran away, if you enable the advanced view on the docker tab you can see if 9c26809c57e9 matches any of your containers - it's the one that spawned the process.

Once you've fixed it, reboot to clear the log so FCP stops warning you.

Looks like maybe you're immich container is pushing your server over the edge, you'll need to add more RAM to the server or limit the amount of RAM your services and VM is allowed.

Reboot to clear the log so FCP stops warning you

Looks like plex pushed it over, limit the memory allowed to the container or reconfigure to cache to disk - reboot to clear the log so FCP stop warning you

You should also try to figure out what's going on here, it's flooding the log
Nov 30 09:09:46 hoth avahi-daemon[2760]: Registering new address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0.*. Nov 30 09:16:46 hoth avahi-daemon[2760]: Withdrawing address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0. Nov 30 09:17:05 hoth avahi-daemon[2760]: Registering new address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0.*. Nov 30 09:26:51 hoth avahi-daemon[2760]: Withdrawing address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0. Nov 30 09:28:06 hoth avahi-daemon[2760]: Registering new address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0.*. Nov 30 09:35:28 hoth avahi-daemon[2760]: Withdrawing address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0. Nov 30 09:36:40 hoth avahi-daemon[2760]: Registering new address record for fdf9:beec:fb34:6c40:XXXX:XXXX:XXXX:f1f9 on br0.*.

Looks like whatever you or your server was doing back on the 11th around 1752 had a java process run away with all of the RAM. If you were doing something at that time (transcodes, immich. whatever is creating all of those docker-proxy processes, etc) you can ignore it, and if it keeps happening you can investigate which one needs to be put on a diet. You can reboot to clear the log so FCP stops warning you.

You've got it right - make sure it's actually transcoding to disk, and if all else fails you can limit the memory allowed to the container.

It could be choking on a particular file, too - judging by how it spikes rapidly (I've had that happen with Plex as well). So you can investigate that angle by narrowing down any recent additions to the library

You need more RAM, 6GB is too little for whatever you're doing - it's killing off smbd which is causing your file browsing issues. Add RAM or try the swap file plugin

Map any cache directories it's using to a disk, /tmp is a common one. If it's a game server, you'll probably just need to throw more RAM at it, turn off other services while it's running, or use the swap plugin

FCP just sees the "Out of Memory" from your log, it's not the host running OOM it's your nVidia card falling over. Try updating drivers/BIOS see if that fixes it or drop it down to gen 4/3

Back on the 5th thru around the 8th one of your containers was hitting its memory limit and was being killed, you can toggle advanced view on the docker tab to find whatever container id this matches to figure out which one it is: d922e3e766ff
If you've already fixed the issue, you can reboot to clear the log to have FCP stop warning you.

Also, NUT appears to be spamming your log, you should fix whatever it's complaining about

Oct 22 17:44:46 T upsmon[3194985]: Poll UPS [[email protected]] failed - Data stale Oct 22 17:44:47 T rc.nut: Writing NUT configuration... Oct 22 17:44:48 T usbhid-ups[3194949]: Reconnecting. If you saw "nut_libusb_get_interrupt: Input/Output Error" or similar message in the log above, try setting "pollonly" flag in "ups.conf" options section for this driver!

Looks like maybe sonarr was doing work that ran the host OOM, you can reboot to clear the log so it stops warning you and if it happens again, check its config or limit the RAM to the container. You also have problems with really long filenames unable to be copied/moved so you should trim them down to a more reasonable length that the filesystem can manage.

Instance:

Sep 21 06:00:07 Tower shfs: copy_file: /mnt/cache/media/Movies/The Devil Comes at Night (2023)/The Devil Comes at Night (2023) [imdb-tt23571868][WEBDL-1080p][8bit][h264][EAC3 5.1]-RABiDS.mkv /mnt/disk23/media/Movies/The Devil Comes at Night (2023)/The Devil Comes at Night (2023) [imdb-tt23571868][WEBDL-1080p][8bit][h264][EAC3 5.1]-RABiDS.mkv.partial (5) Input/output error

Looks like your LLM container hit its limit back on the 26th and was killed. Reboot to clear the log so FCP stops warning you

Probably not related

Go to the docker tab and toggle the advanced view and see which container has this ID (or close to it) 7d3934da0c4f - it may be nginx proxy manager as another user recently had the same node process running way going OOM and being killed. I can't offer any advice as to why it's doing that, I have been running it for years and have never seen it do that, best I can advise is to re-install it.

You will need to restart the sever to clear the log

You should also try to figure out what's going on with these, it's flooding your log

Oct 3 14:37:35 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is reachable again Oct 3 14:47:46 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is unreachable Oct 3 14:47:46 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is reachable again Oct 3 14:47:55 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is unreachable Oct 3 14:48:40 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is reachable again Oct 3 14:48:48 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is unreachable Oct 3 14:49:22 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is reachable again Oct 3 14:49:30 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is unreachable Oct 3 14:49:30 Tower dhcpcd[1976]: br0: fe80::be24:11ff:fe4c:4bc3 is reachable agai

Looks like syncthing hit the container's memory limit and was killed

Your CCTV drive is dying for sure, for the OOM it looks like you've got a lot going on for 32GBs, re-allocate where you can and limit your containers

Your container is hitting the memory limit and being killed, and the system is reporting that


Runps -auxf > ps.txt to pipe it to a file (run within a share path so it will create the file there) that will show you what is launching the python process that's causing the issue.

Looks like it happened back on the 19th of August, there were a few plex transcodes in process so if you have those going to RAM then that contributed to it. When the host went OOM the reaper killed your home assistant VM to keep the host up, but doesn't look like anything to worry about. Reboot to clear the log so FCP stops warning you and if it happens again you can limit the memory allowed to your containers/VMs.

Power delivery issue looks likely, if you're running splitters - don't.

Whatever container you have running that python instance is hitting its memory limit and is being culled. Frigate also caused the host to OOM around the same time and was killed by the kernel, you should configure the cache for the transcode to go to disk.

When you've resolved it, you can restart to clear the log so FCP doesn't keep warning you.