I am experiencing multiple Unraid hanging, no response, unresponsive.

No error messages printed to console.

Cannot log into console either.

I have attached three logs.

I have since rebooted and attached the diagnostic logs.

 

Thank you.

 

 

 

tower-diagnostics-20241015-1640.zip tower-diagnostics-20241013-2009.zip tower-diagnostics-20241012-2056.zip

Enable the syslog server and post that after a crash.

Gathered a log for the current "hang"... I cannot access Unraid WebUI.  Cannot access via SSH.  Cannot access any dockers.

 

The problem occurs around Oct 16 17:38:12.

<6>Oct 16 17:27:18 Tower kernel: docker0: port 5(veth34567ba) entered disabled state
<6>Oct 16 17:27:18 Tower kernel: device veth34567ba entered promiscuous mode
<6>Oct 16 17:27:25 Tower kernel: eth0: renamed from veth9e3327d
<6>Oct 16 17:27:25 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth34567ba: link becomes ready
<6>Oct 16 17:27:25 Tower kernel: docker0: port 5(veth34567ba) entered blocking state
<6>Oct 16 17:27:25 Tower kernel: docker0: port 5(veth34567ba) entered forwarding state
<6>Oct 16 17:27:27 Tower kernel: docker0: port 5(veth34567ba) entered disabled state
<6>Oct 16 17:27:27 Tower kernel: veth9e3327d: renamed from eth0
<6>Oct 16 17:27:27 Tower kernel: docker0: port 5(veth34567ba) entered disabled state
<6>Oct 16 17:27:27 Tower kernel: device veth34567ba left promiscuous mode
<6>Oct 16 17:27:27 Tower kernel: docker0: port 5(veth34567ba) entered disabled state
<6>Oct 16 17:28:28 Tower kernel: docker0: port 5(vethaaffec7) entered blocking state
<6>Oct 16 17:28:28 Tower kernel: docker0: port 5(vethaaffec7) entered disabled state
<6>Oct 16 17:28:28 Tower kernel: device vethaaffec7 entered promiscuous mode
<6>Oct 16 17:28:36 Tower kernel: eth0: renamed from vethb77cf64
<6>Oct 16 17:28:36 Tower kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethaaffec7: link becomes ready
<6>Oct 16 17:28:36 Tower kernel: docker0: port 5(vethaaffec7) entered blocking state
<6>Oct 16 17:28:36 Tower kernel: docker0: port 5(vethaaffec7) entered forwarding state
<6>Oct 16 17:28:38 Tower kernel: vethb77cf64: renamed from eth0
<6>Oct 16 17:28:38 Tower kernel: docker0: port 5(vethaaffec7) entered disabled state
<6>Oct 16 17:28:38 Tower kernel: docker0: port 5(vethaaffec7) entered disabled state
<6>Oct 16 17:28:38 Tower kernel: device vethaaffec7 left promiscuous mode
<6>Oct 16 17:28:38 Tower kernel: docker0: port 5(vethaaffec7) entered disabled state
<4>Oct 16 17:38:12 Tower kernel: md: sync done. time=89664sec
<4>Oct 16 17:38:17 Tower kernel: md: recovery thread: exit status: 0
<35>Oct 16 19:22:10 Tower sshd[11363]: error: beginning MaxStartups throttling

 

There does appear to be activity in the log, so the server is not dead.

 

During this time, parity is being validated (because of a previous hang which resulted in an unclean shutdown), but I cannot see messages. 

 

I also have BTRFS activity for moving data off one of my drives (I need to free-up a drive bay to replace with a larger drive).  This is not the unraid array, but a separate BTRFS pool of old drives).  The current hanging problem happened before, so I don't think this is causing a problem.  I can still see BTRFS move messages in the syslog, so that's how I know Unraid is partially working.  This event appears to be the only thing working right now.

 

--- just as I was writing this, the BTRFS activity completed (good news), and I can see it has completed in the original SSH console that I issued the command  However, the SSH console is not responsive to my keyboard input, so I cannot do anything else. 

 

Unfortunately WebUI doesn't work, nor does SSH.  The console allows me to type in the username login, but then timeouts with a timeout message, and I can't type a password or login through the console.  So again, there's some sort of activity going on, but something appears to be not allowing logins or the server to make connections.

 

I'm curious if one of my docker containers is taking down the OS.  I don't know if that can happen, or figure out which container.

 

 

syslog.txt

If the server keeps running it suggests more an apps issue, one thing you can try is to boot the server in safe mode with all docker containers/VMs disabled, let it run as a basic NAS for a few days, if it doesn't crash like that, start turning on the other services one by one, including the docker containers.