The Odd Angry Hang or Restart

[dalben]

My UnRAID server is for the most part quite solid, but I do get the occasional restart or server hang.  Nothing has ever popped up in the syslog before the event, and as it's headless I've never seen what the screen is showing.  I can go a couple of months without anything or I can have a few in a short space of time.

 

Yesterday I had a lockup where a cold restart was needed to get back.  I just had a server reboot.  Because it's not constant, or can be triggered, I've refrained from trial by elimination and starting it without plugins or dockers and slowly adding until I get an event.  It would take too long and render my server useless for what I need it for.  So for now it while just be something to deal with but I've attached my diagnostics in case someone has the desire to wade through and see if there's anything obvious amiss.  There's a few errors in the syslog at every startup but I've always ignored those without knowing what they mean.

 

TIA to anyone that has the time to have a look.

tdm-diagnostics-20200525-1502.zip

[JorgeB]

Diags after rebooting aren't much help, you can try this:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=781601

 

[dalben]

1 hour ago, johnnie.black said:

Diags after rebooting aren't much help, you can try this:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=781601

 

I do run syslog onto a server as well as duplicate it on flash. This was in the attempt to capture some info before the boot/hang. There is nothing before a hang or reboot that I see. 

 

Let me grab the current syslog and anonymise it a bit and I can attach. I mistakenly thought the Diags would grab the whole syslog. 

Edited May 25, 2020 by dalben

[dalben]

Here's the syslog entries leading up to a reboot or hang.  As you can see there is nothing in the logs pointing to why the server has hung or rebooted itself.  Happy to send through the whole syslog if that helps.

Mar 30 10:31:37 tdm kernel: mdcmd (467): spindown 1
Mar 30 11:27:00 tdm kernel: mdcmd (468): spindown 1
Mar 30 12:22:15 tdm kernel: mdcmd (469): spindown 3
Mar 30 12:27:56 tdm kernel: mdcmd (470): spindown 1
Mar 30 14:21:44 tdm kernel: mdcmd (471): spindown 1
Mar 30 17:25:52 tdm kernel: microcode: microcode updated early to revision 0x2f, date = 2019-02-17

Apr  4 18:40:44 tdm kernel: mdcmd (174): spindown 2
Apr  4 18:52:51 tdm kernel: mdcmd (175): spindown 1
Apr  4 18:58:48 tdm kernel: mdcmd (176): spindown 3
Apr  4 19:54:27 tdm kernel: mdcmd (177): spindown 2
Apr  4 19:55:46 tdm kernel: mdcmd (178): spindown 3
Apr  4 20:12:58 tdm kernel: microcode: microcode updated early to revision 0x2f, date = 2019-02-17

May  2 20:43:11 tdm kernel: mdcmd (294): spindown 6
May  2 20:44:34 tdm kernel: mdcmd (295): spindown 3
May  2 20:45:37 tdm kernel: mdcmd (296): spindown 5
May  2 21:07:43 tdm kernel: mdcmd (297): spindown 2
May  2 21:07:54 tdm kernel: mdcmd (298): spindown 3
May  2 21:25:36 tdm kernel: mdcmd (299): spindown 6
May  2 21:28:26 tdm kernel: mdcmd (300): spindown 2
May  2 21:28:54 tdm kernel: mdcmd (301): spindown 3
May  2 21:31:50 tdm kernel: microcode: microcode updated early to revision 0x2f, date = 2019-02-17

May 24 08:03:20 tdm kernel: mdcmd (1394): spindown 2
May 24 08:18:52 tdm kernel: mdcmd (1395): spindown 3
May 24 11:20:01 tdm kernel: mdcmd (1396): spindown 1
May 24 12:29:08 tdm kernel: mdcmd (1397): spindown 2
May 24 12:29:08 tdm kernel: mdcmd (1398): spindown 5
May 24 14:06:54 tdm kernel: microcode: microcode updated early to revision 0x2f, date = 2019-02-17

May 25 12:47:05 tdm kernel: mdcmd (73): spindown 3
May 25 13:13:55 tdm kernel: sd 9:0:6:0: [sdi] tag#2606 UNKNOWN(0x2003) Result: hostbyte=0x00 driverbyte=0x00
May 25 13:13:55 tdm kernel: sd 9:0:6:0: [sdi] tag#2606 CDB: opcode=0x88 88 00 00 00 00 01 5e db 0b 80 00 00 00 08 00 00
May 25 13:13:55 tdm kernel: print_req_error: I/O error, dev sdi, sector 5886380928
May 25 13:40:29 tdm kernel: mdcmd (74): spindown 6
May 25 13:55:12 tdm kernel: mdcmd (75): spindown 5
May 25 13:59:19 tdm kernel: mdcmd (76): spindown 6
May 25 13:59:43 tdm kernel: mdcmd (77): spindown 2
May 25 14:17:15 tdm kernel: microcode: microcode updated early to revision 0x2f, date = 2019-02-17

 

[JorgeB]

That suggests a hardware issue, but not easy to diagnose if sometimes it takes so long to crash.

[dalben]

Yeah, it's an annoying one. There's a few errors being thrown in the start up that point to hardware but I don't know enough to understand whether they are important or can be ignored.