[6.8.3] Cannot reach unraid.

[Bleak]

So for the last two days my unraid becomes unresponsive about every two to 3 hours only solution is a hard shutdown (holding the power button).

I am unable to hook up a monitor to it so not sure if it is a network thing or the system hangs itself?

 

It used to work fine and never had any major issues.

 

I have part of a syslog here (with the last entry at 00:35:18 before it became unresponsive and I restarted it hours later) but no clue if any of this clarifies why I am having this issue any help is welcome.

Jun 26 23:48:25 Zeus rsyslogd: [origin software="rsyslogd" swVersion="8.1908.0" x-pid="31503" x-info="https://www.rsyslog.com"] start
Jun 26 23:48:53 Zeus ntpd[2043]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Jun 26 23:53:05 Zeus sshd[5142]: Accepted password for root from 192.168.1.10 port 58976 ssh2
Jun 26 23:54:00 Zeus root: Fix Common Problems Version 2020.05.05
Jun 26 23:54:12 Zeus root: Fix Common Problems: Warning: Syslog mirrored to flash
Jun 27 00:35:17 Zeus kernel: veth5490b91: renamed from eth0
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(vethd3da599) entered disabled state
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(vethd3da599) entered disabled state
Jun 27 00:35:17 Zeus kernel: device vethd3da599 left promiscuous mode
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(vethd3da599) entered disabled state
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(veth3fcd9d0) entered blocking state
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(veth3fcd9d0) entered disabled state
Jun 27 00:35:17 Zeus kernel: device veth3fcd9d0 entered promiscuous mode
Jun 27 00:35:17 Zeus kernel: IPv6: ADDRCONF(NETDEV_UP): veth3fcd9d0: link is not ready
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(veth3fcd9d0) entered blocking state
Jun 27 00:35:17 Zeus kernel: br-aff798abb7c1: port 9(veth3fcd9d0) entered forwarding state
Jun 27 00:35:18 Zeus kernel: eth0: renamed from vethb75b213
Jun 27 00:35:18 Zeus kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth3fcd9d0: link becomes ready
Jun 27 12:56:42 Zeus kernel: Linux version 4.19.107-Unraid (root@38721b48cdfb) (gcc version 9.2.0 (GCC)) #1 SMP Sun Mar 8 14:34:03 CDT 2020
Jun 27 12:56:42 Zeus kernel: Command line: BOOT_IMAGE=/bzimage isolcpus=0-1,8-9 initrd=/bzroot
Jun 27 12:56:42 Zeus kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Jun 27 12:56:42 Zeus kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Jun 27 12:56:42 Zeus kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Jun 27 12:56:42 Zeus kernel: x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Jun 27 12:56:42 Zeus kernel: x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
Jun 27 12:56:42 Zeus kernel: BIOS-provided physical RAM map:
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x0000000000100000-0x0000000009d7ffff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x0000000009d80000-0x0000000009ffffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x000000000a000000-0x000000000a1fffff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x000000000a200000-0x000000000a20afff] ACPI NVS
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x000000000a20b000-0x000000000affffff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x000000000b000000-0x000000000b01ffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x000000000b020000-0x00000000dbc62fff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000dbc63000-0x00000000dbdd1fff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000dbdd2000-0x00000000dc254fff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000dc255000-0x00000000dc36cfff] ACPI NVS
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000dc36d000-0x00000000dcfd9fff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000dcfda000-0x00000000deffffff] usable
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000df000000-0x00000000dfffffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x00000000fd000000-0x00000000ffffffff] reserved
Jun 27 12:56:42 Zeus kernel: BIOS-e820: [mem 0x0000000100000000-0x000000081f37ffff] usable
Jun 27 12:56:42 Zeus kernel: NX (Execute Disable) protection: active
Jun 27 12:56:42 Zeus kernel: e820: update [mem 0xca4df018-0xca4ed057] usable ==> usable
Jun 27 12:56:42 Zeus kernel: e820: update [mem 0xca4df018-0xca4ed057] usable ==> usable

Will try to get a diagnostics file but it is difficult to get one..

Edited June 27, 2020 by Bleak

[Bleak]

Happened to notice that one port was used by two containers changed that not sure if that fixed it but has been running fine since this post.

 

[Frank1940]

Read this on how to set up the syslog server and see if it captures anything that might be a clue.

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/?tab=comments#comment-781601

 

Since it is such a short time to a problem I would suggest using the flash/boot drive.

 

[Bleak]

It has not happened since I stopped the container it was the Unifi controller docker not sure if it was the ports or something else but will try it again later whenever I have some free time to deal with it. Thanks for your link will certainly help with future troubleshooting.

[Bleak]

Okay so It is happening again so turned on the syslog again. I mirrored it to flash since I do not want to turn it back on again until I need to or know how to fix it cause me having to do a hard reboot cannot be healthy to the system. please see attached the syslog you can see that the syslog ends at 17:49:24 on july 5th

and it starts again when I do the hard reboot at 19:14:16.

 

Hope someone can point me in the right direction...

syslog(1)

[JorgeB]

Unfortunately nothing logged that points to the issue, could be a hardware problem, since it's a Ryzen CPU have you seen this?

[Bleak]

Okay went in the bios (Just note it has been stable on these bios settings for at least half a year until a few weeks ago) noticed my bios was flickering and bit unresponsive.. Decided to replace the CMOS battery since this motherboard is getting older not sur eif replacing the cmos battery or just the clear cmos fixed the bios flickering but one of the two things did it's job.

 

Re applied all the settings and for some reason it now has been stable for over 24 hours... keeping my fingers crossed for now but I really hope this nightmare is over. Next unraid system is not going to be a Ryzen system anymore it might be fun for desktop but that is all I will consider it for in the future..

 

Edit: has been stable for almost a week now issue is solved.

Edited July 10, 2020 by Bleak