Over the last 2 weeks or so there have been 2-3 times where i would go to use one of my self-hosted services to find they didnt respond. Try to access the unraid gui and it wouldnt respond, couldnt ssh in, nothing. I end up having to hard power it off and then things are fine again for a while but since i cant ssh in i cant get diagnostics. This morning i had a similar issue but after 5+ minutes it finally loaded and i was able to grab diagnostics. I have zero idea if this is a related issue or not but i was hoping the diagnostics would point you guys in the direction of potential issues maybe.

Thanks!

storage-diagnostics-20251118-0743.zip

There's a segfault, but it was earlier.

Nov 18 06:45:46 Storage kernel: UPID:Storage:00[1855345]: segfault at 30 ip 000014e21dbdb615 sp 000014e183bfbdb0 error 4 in libapt-pkg.so.7.0.0[1db615,14e21da3d000+1cf000] likely on CPU 36 (core 9, socket 1)

Nov 18 06:45:46 Storage kernel: Code: 83 ec 28 48 8b 15 2b 98 09 00 64 48 8b 1c 25 28 00 00 00 48 89 5c 24 18 48 89 fb 48 c7 47 08 00 00 00 00 48 83 c2 10 48 89 17 <48> 8b 56 30 0f b7 6a 36 48 89 73 10 48 c7 47 18 00 00 00 00 48 c7

Nov 18 06:54:22 Storage webgui: docker stats error:

Then the mover still ran, but could still be related.

@JorgeB

Im not sure what the segfault message means, does that mean its a memory issue or a cpu issue since it references cpu 36?

I forgot i had setup remote syslog streaming to another unraid box. Also, it happened again and i had to hard power it so i grabbed a large chunk of the syslog and attached it here so maybe that will help!

syslog.txt

Not always easy to tell, and from a single one, even more, I see if there are more when it happens again, you can enable the syslog server, in case you lose the GUI.

yeah, i have the remote syslog server enabled, thats how i got the syslog chunk i uploaded 30 minutes ago. Was there anything in that piece that gave you any insight?

I see some of these:

Nov 18 12:09:28 Storage php-fpm[11257]: [WARNING] [pool www] child 2312909 exited on signal 9 (SIGKILL) after 275.582856 seconds from start
Nov 18 12:09:28 Storage php-fpm[11257]: [WARNING] [pool www] child 2312909 exited on signal 9 (SIGKILL) after 275.582856 seconds from start
Nov 18 12:13:21 Storage php-fpm[11257]: [WARNING] [pool www] child 2312985 exited on signal 9 (SIGKILL) after 251.371273 seconds from start
Nov 18 12:13:21 Storage php-fpm[11257]: [WARNING] [pool www] child 2312985 exited on signal 9 (SIGKILL) after 251.371273 seconds from start
Nov 18 12:19:23 Storage php-fpm[11257]: [WARNING] [pool www] child 2313109 exited on signal 9 (SIGKILL) after 256.520379 seconds from start
Nov 18 12:19:23 Storage php-fpm[11257]: [WARNING] [pool www] child 2313109 exited on signal 9 (SIGKILL) after 256.520379 seconds from start

In my experience, these errors can be the result of the server being close to exhausting the memory, GUI can become extremely slow, like 1 minute to open the dashboard, try limiting the memory for VMs/docker services, or adding a little more RAM.

It could also be one or more containers hogging the CPU, try pinning only some cores to them, and leave cores 0/1 available for Unraid.

Also, recommend trying a couple of other things, go to Settings - Global share settings and set the Number of fuse File Descriptors to the max, and enable this:

https://docs.unraid.net/unraid-os/release-notes/7.0.0/#excessive-flash-drive-activity-slows-the-system-down