Greetings,

 

I was trying to use plex this morning, and it wouldn't load. I checked unraid, and it was very sluggish, and one of my cores was pegged at 100%. Which is very unusual. I checked the system log, and found numerous entries such as this:

Aug 31 07:08:01 UnRaid kernel: Call Trace:
Aug 31 07:08:01 UnRaid kernel: __schedule+0x45e/0x4a4
Aug 31 07:08:01 UnRaid kernel: schedule+0x73/0x99
Aug 31 07:08:01 UnRaid kernel: schedule_timeout+0xa7/0xe0
Aug 31 07:08:01 UnRaid kernel: ? __next_timer_interrupt+0xe8/0xe8
Aug 31 07:08:01 UnRaid kernel: rcu_gp_kthread+0x417/0x7a9
Aug 31 07:08:01 UnRaid kernel: ? kfree_rcu_work+0x19e/0x19e
Aug 31 07:08:01 UnRaid kernel: kthread+0xe5/0xea
Aug 31 07:08:01 UnRaid kernel: ? __kthread_bind_mask+0x57/0x57
Aug 31 07:08:01 UnRaid kernel: ret_from_fork+0x22/0x30
Aug 31 07:09:01 UnRaid kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
Aug 31 07:09:01 UnRaid kernel: rcu: 10-...!: (0 ticks this GP) idle=f4c/0/0x0 softirq=58410444/58410444 fqs=2 (false positive?)
Aug 31 07:09:01 UnRaid kernel: (detected by 22, t=60002 jiffies, g=143930057, q=204228)
Aug 31 07:09:01 UnRaid kernel: Sending NMI from CPU 22 to CPUs 10:
Aug 31 07:09:01 UnRaid kernel: NMI backtrace for cpu 10 skipped: idling at acpi_idle_do_entry+0x31/0x37
Aug 31 07:09:01 UnRaid kernel: rcu: rcu_sched kthread starved for 60003 jiffies! g143930057 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=10
Aug 31 07:09:01 UnRaid kernel: rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.
Aug 31 07:09:01 UnRaid kernel: rcu: RCU grace-period kthread stack dump:
Aug 31 07:09:01 UnRaid kernel: task:rcu_sched state:I stack: 0 pid: 10 ppid: 2 flags:0x00004000

 

I don't know linux good enough to determine what the cause of this could be? I tried to kill the vm that is pinned to cpu 10/22(which is it's physical and logical core) and that did not help. It actually made things worse. I ended up having to hard reset the box.

 

I've also have come in to work in the morning and plex was down. And checking unraid, the whole box was unresponsive. syslog never showed me anything out of the ordinary. the last few lines were just things like a drive spinning down, or the fans spinning up or down due to drive temps. I think I may have been running in to this issue for a few weeks now, and only now was able to see it happening in real time to see better data?  I can grab splunk logs if that would help as I am willing to bet the diagnostic zip won't have this data in it, as I had to reboot to get back in to unraid. I was unable to pull logs prior as the ui was basically unresponsive.. 

 

Any help would be greatly appreciated here! Thanks!!!

unraid-diagnostics-20210831-0808.zip

Just following up, that I am again running in to where the server will lock up during the night at some point. I am a bit surprised that I have not had any responses? Did I stump everybody??

seems like it ... i'm having thesame issues.

Still having the issue. Just had it lock up again last night. 

Me too ... plagued by cpu stalls daily. Box becomes slowly unresponsive, syslog fills with garbage and ultimately requires hard reset. Just about had enough of it now and considering changing out all hardware, mobo, cpu, ram cos its becoming a real pita! Thereagain it can go for days not needing any attention as it used too but it seems any attempt to actually USE the machine triggers a reaction. Not a lot of good really just having it stand idle ... anyone with any ideas on how to troubleshoot this would be welcome to chip in ...

Yep, it has been up for over 3 days now with no issue. It can't go a week or more without locking up, and then boom. I wake up and it is hard locked up. I run pi-hole on a vm, and so I have no internet basically when this happens... It could then lock up again in the same day, or next morning, etc. and then go back to working fine for days. It is completely random. I have not been able to identify anything that could be contributing. I have a new mobo, but would like to get a new proc before putting it in. I just don't know anymore. It's super frustrating. Checking hardware shows no issues too.