This morning was an interesting event. My server has been running flawlessly for months and then this morning I wake up to see my server can't be accessed via IP. I go into the console and I'm able to login. I had to get the server up and running so I pulled the syslog real quick and had to manually reboot (shutdown command hung).

 

In the syslog, I see that there are several entries for stalls

 

Apr 22 08:57:38 Smalls kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Apr 22 08:57:38 Smalls kernel: rcu:     16-....: (59999 ticks this GP) idle=b76/1/0x4000000000000000 softirq=103787593/103787593 fqs=14982 
Apr 22 08:57:38 Smalls kernel:     (t=60000 jiffies g=840289921 q=130361)
Apr 22 08:57:38 Smalls kernel: NMI backtrace for cpu 16

 

The most recent changes made to this system are:

1) Installation of a PiHole docker container

2) Moving the DAC from one port to the other on the 10 gig network card yesterday (was working fine all day afterwards)

3) Removed the PiHole docker container this morning after the reboot since I was no longer using it

 

Diagnostics were pulled after the reboot. Unfortunately I couldn't find the command line to run for diagnostics before I had to get the server back up.

 

I guess my question is, if this happens again:

1) What is the command line to pull and anonymize diagnostics?

2) With the syslog being pulled before reboot, is there anyone who might know what the issue might be?

 

Thanks in advance!

syslog smalls-diagnostics-20220422-1125.zip

5 minutes ago, OrneryTaurus said:

What is the command line to pull and anonymize diagnostics?

explained in the diagnostics link

 

 

2 minutes ago, trurl said:

explained in the diagnostics link

 

 

 

Of course it would be that simple lol. Totally overlooked that when scrambling. Thanks! I'll keep this in mind for what (hopefully) won't be a next time.