GUI not accessible after reboot due to 95% rootfs full

[Boo-urns]

So I've just done a reboot and now the server UI isn't accessible at all. Device doesn't show up in router devices (has a static IP set in Unraid).

 

Boot screen (black screen/white text) gets to tower login, but GUI doesn't load. I did notice it mentioned eth0 not found during the boot sequence whether that affects it.

 

Not sure what has broken it, but any suggestions to troubleshoot? I've tried safe mode with/without GUI with no luck.

 

Prior to rebooting I had my rootfs 95% used, and high RAM usage, which I think was due to Plex transcoding to /tmp. I cleared the variable and rebooted to clear RAM and now I'm stuck. Any ideas?

[Squid]

If it's showing Tower Login and the wall of text is still there, then you booted normally, and not via the GUI mode.  GUI mode is an option from the webGUI over the network

 

login with root and your password and then enter in 

diagnostics

If you can reach your shares over the network, then grab the diagnostics file that's saved on the flash share in the logs folder and post it here

 

If not, then

powerdown

pull the flash toss it into another system and post it here

[Boo-urns]

2 hours ago, Squid said:

If it's showing Tower Login and the wall of text is still there, then you booted normally, and not via the GUI mode.  GUI mode is an option from the webGUI over the network

 

login with root and your password and then enter in 

diagnostics

If you can reach your shares over the network, then grab the diagnostics file that's saved on the flash share in the logs folder and post it here

 

If not, then

powerdown

pull the flash toss it into another system and post it here

logged in with root and ran "diagnostics". Saved file attached. 

Was not available over the network so this was taken directly from flash.

Cheers @Squid. Server is unusable currently so any help would be appreciated.

skynet-diagnostics-20210422-2304.zip

[Boo-urns]

@Squid Additionally, booting to GUI Mode (and GUI Safe Mode) shows the login screen, but i'm unable to enter text to try and login.

[Boo-urns]

OK, randomly after a few reboots it magically showed up in router devices and I could login to the webGUI. I thought perhaps my previous pfsense VM could be contributing, so I disabled VMs and set PCIe ACS Override back to disabled. I rebooted and once again network was down during boot and unreachable.

I did notice that the network activity light on the mobo LAN connection intermittently turned off completely, where otherwise it staying flashing Gb connection. So I thought perhaps faulty cable? Replaced cable no change. Faulty LAN port maybe?

 

I restored a previous flash backup, which didn't actually change anything because it updated to Unraid.net during the brief period it was working. But it booted somehow.

I created a bond between my mobo LAN port (eth0) and my 4 port PCIe NIC (eth1) and made them redundant (mode 1), so I could test whether it was the mobo LAN port, but since then I haven't had it fail.

 

Attached are current diagnostics with Unraid working, and obviously the above diags are from when unraid was not booting (network down).

 

The only thing I can think of is if the motherboard ethernet port was intermittently failing, in which case i'd be looking to RMA as it's only a few months old. However if it's an Unraid issue i'd love to know what was causing it so I can avoid this in future.

 

If you can make sense of the logs it would be appreciated.

skynet-diagnostics-20210424-1103 (working).zip