I was watching a movie last night with the family and all of a sudden Plex stopped playing the movie. A few moments later it looked like it was down.

I tried to log into the web page of the server but it was not responding. I also could not ssh into it either. However, I could ping it.

I went into the basement and connected a monitor and keyboard. I could log in and saw that BOINC was using lots of memory. The system was very slow at this point and it took a while for "top" to even come up. I paused the BOINC container and it started to behave normally and we finished the movie.

This morning I see in the syslog a bunch of oom-killer processes for various apps on my server. These go all the way back to 10/4/2025 of this year. That day also happens to be the first time I got an out of memory error. It bombed on postgres but I was also running a VM with 8GB of RAM at the time so I think that's what did it. It's running a monthly parity check as I type this.

Not sure what's going on this time and it seems to be happening often from what I can tell in the logs. Not sure why I was OK from 12/27/2024 to 10/4/2025.

Maybe I just need more RAM? I'm on 32GB and my old server had 64GB.

flores-diagnostics-20251201-1149.zip

Edited December 1, 2025Dec 1 by nraygun

Still looking through the logs but I see that things seemed to start going haywire on Friday 11/28/2025 at 22:35:05. It went almost a month with no oom-killing. And before 11/28, the last one was on 10/4/2025, and then before that it was 12/27/2024.

On 11/28 it started to call oom-killer for different processes every few minutes! I haven't had any today on 12/1/2025.

Got a few theories:

1. Maybe BOINC was causing all this. I see in the config the memory usage was set to use up to 90%. The client is not using the setting I have setup via web prefs. I manually set it to 50%.

2. Bad memory? As soon as it's done with the parity check, I'll run a Memtest.

ontainerd-shim being the trigger strongly suggests that it was a container causing the issue.

12 hours ago, nraygun said:

Maybe BOINC was causing all this. I see in the config the memory usage was set to use up to 90%. The client is not using the setting I have setup via web prefs. I manually set it to 50%.

Probably, since it was einstein that was being killed repeatedly back on the 29th and 30th.

12 hours ago, nraygun said:

Bad memory? As soon as it's done with the parity check, I'll run a Memtest.

Bad memory wouldn't cause an OOM, only using too much RAM will.

Fix whatever the container is doing, or limit the memory available to the container itself, and reboot to clear the log.

Yep, it was BOINC.

I manually started it and the RAM usage jumped to 94% after a minute or two.

There must have been an update that broke it.

I'll leave it off for a day or two to see if I continue to get the OOM messages.

Yep, BOINC was the problem. Relatively clean syslog now with no odd behaviors.