Sever suddenly crashing daily

[keitaro26]

I am hoping someone can help me in the right direction. Recently my server has been crashing every day. It doesn't seem to follow a logical process; sometimes it will be online nearly 24 hours, sometimes it crashes in 20 minutes. I've tried disabling all my docker containers, reinstalling my plugins, but nothing seems to help. I've attached my diagnostics, but I am not really sure what I am looking for in them.

magi-diagnostics-20240804-0045.zip

[itimpi]

The syslog in the diagnostics is the RAM copy and only shows what happened since the reboot.   It could be worth enabling the syslog server to get a log that survives a reboot so we can see what happened prior to the reboot. The mirror to flash option is the easiest to set up, but if you are worried about excessive wear on the flash drive you can put your server’s address into the Remote Server field.

[keitaro26]

I've enabled the syslogs. I will reply with them when it crashes next.

[keitaro26]

Just crashed. Here is the log. I don't see anything helpful in it. 

syslog-192.168.1.110.log

[JorgeB]

Unfortunately there's nothing relevant logged, this can be a hardware issue, one thing you can try is to boot the server in safe mode with all docker containers/VMs disabled, let it run as a basic NAS for a few days, if it still crashes it's likely a hardware problem, if it doesn't start turning on the other services one by one.

[keitaro26]

Still crashing with everything turned off and the graphics cards removed. Seems likely that its MB or CPU, so I am going to swap those out for a spare Intel set I have and see if that fixes things. This is the first time I am changing my MB and CPU on Unraid, any gotcha's I should be aware of? it will be a i5-12600k and a ASROCK Z690 PRO RS I am testing with.

[keitaro26]

Got the new board installed and was running a parity check for many hours before it soft locked. My dockers are still accessible, but I can't get to the main interface. Here is the syslog, can someone interpret for me?

syslog-192.168.1.110.log

[JorgeB]

There's a segfault and a call trace logged, did you run memtest?

[keitaro26]

I did not run one since replacing the motherboard and CPU. When it was crashing prior, I did run one with no errors.

[JorgeB]

Since memtest is only definitive if it finds errors, if you have multiple sticks try using the server with just one, if the same try with a different one, that will basically rule out bad RAM.