(SOLVED) rcu_sched self-detected stall on CPU

[rvoosterhout]

Hello,

 

Since a few days, my server is crashing when my mover starts. Since the problem started, I had my Unraid server export to a syslog server, so attached you will find my syslog file and diagnostics. The server crashes at 12/14/2019 22:08 (mover starts at 22:00).

Unraid version: 6.8.0

Cache disk: xfs

Array disk (5): xfs

 

When the server hangs, i can still access the command line using my iDrac on my R710, but when I issue a powerdown -r, the server starts it reboot cycle, but hangs at 'waiting 120 seconds to gracefully shut down'. Then it's starts to collect diagnostic files, but it won't get any further than that. A warm power cycle is the only option to recover.

 

I did already read this topic: 

But in the comment mentioned above, that issue should have been solved in the 6.0 beta-15 update.

 

2019-12-14.csv : Syslog file, server crash at 12/14/2019 22:08

 

Any help would be kindly appreciated.

 

Rick

2019-12-14.csv behemoth-diagnostics-20191214-2234.zip

Edited December 30, 2019 by rvoosterhout

[JorgeB]

This looks more like a hardware problem, but did it start after upgrading Unraid? If you downgrade to v6.7.2 does it go away?

[rvoosterhout]

I'm not sure, i'll downgrade and see if I still have the issue.

[rvoosterhout]

So after starting the reboot for the rollback, the server hangs again. Again a warm reboot is necessary get any response.

 

Reboot issued at 12/15/2019 11:42:29

 

I am running version 6.7.2 now.

2019-12-15.csv behemoth-diagnostics-20191215-1144.zip

[rvoosterhout]

Rebooted manually again. No issues this time, reboot was succesfull.

 

I'll upload a big file to my server and then start the mover to see what happens.

[rvoosterhout]

With version 6.7.2 I don't have any problems with running my mover and rebooting the server. Could the 6.8 update be the cause?

[JorgeB]

Could be some something specific to your config, try again with v6.8 with VM/docker services disable and in safe mode to rule out any plugins.

[rvoosterhout]

I updated back to 6.8, and checked my cache drive using xfs_repair. The drive did have some issues, which are fixed. Now the server is running good.