Server hanging during boot

[bhinkle50]

I woke up this morning to a VM that had no ethernet. I rebooted the server to see if it would clear it up and it had same issue once it came back up.

 

I went into network settings and it showed that one of my ports was down and it required me to shut down Docker and VMs to edit network settings to remove that port. I made a change to disable the port, but the server didn't appear to take the save and then hung. 

 

I did a hard power down and the server isn't coming back up. It appears to keep getting stuck on the mount of disks, but it isn't consistent which disk it stops on. I have pulled all cables and reseated them to make sure it's not a cable issue.

 

I'm currently trying to reboot in safe mode and it was stuck at the step in the photo for 15 min. I have since powered off and pulled the diagnostics from the last successful boot.

 

Any thoughts are appreciated.

 

Thanks.

hydra-diagnostics-20230517-1114.zip

[JorgeB]

Looks more like a server hardware problem, but also try a new flash drive with a stock Unraid install, no key needed, just to rule out any flash or config issues.

[bhinkle50]

So this was a pretty weird one. It started in the AM with my server losing it's network connection. I also experienced some other networking issues which caused me to reboot the network and at the same time I rebooted my server. The server began to hang at the boot steps above, so I believed the issue was all related to my server. Then I had to reboot my network again, and again, and again. Now I can't get my server to boot, and my router appears to be going out.

 

I pulled every piece of hardware out of the server. 17 drives and 4 exp cards. I then went one by one. It started booting!!!! 

 

With that success, I slowly added pieces back into the mix. I eventually got it to boot fully - ran a parity check, backed up VMs, appdata, and flash. I then rebooted and it STOPPED again, but this time the flash didn't even load to get to the boot. Restarted again, and it got to a different part before it stopped. 

 

Then it dawned on me that my flash is obviously failing. So I swapped out the flash drive, booted perfectly, I transferred the key and it has been rock solid. Well, until I started messing with the VMs in prep for my migration and screwed it all up. But that's another thread that you can see here.

 

Oh, almost forgot the best part. The original networking issue? It was because ASUS sucks at devops. They released a security file to their routers that consumed all the memory and caused routers to consistently crash. They suck and you can read about it here.

Edited May 25, 2023 by bhinkle50