Remote server not coming back up after 6.5.1 update and reboot


Recommended Posts

Hi there.  I tried updating my remote Plex server (JBOX in my sig) from 6.5.0 to 6.5.1 but rebooting to complete it the UI won't come back up again and I can't SSH in via putty either.  Array auto-start is disabled and for the moment no one is home to check if it actually rebooted (ie. power light on) though I can't imagine it wouldn't have.  So for now the only theory I have is maybe the DHCP reservations on the router got changed and the server was assigned a different IP address than I have bookmarked?  Or would unRAID still assign the last IP from the Network Settings even if the reservation on the router had been deleted?  Any other suggestions on what I can do to try to regain control?  I have a Win 10 computestick there that I use as a control via Chrome remote desktop and the only thing I can't access through that is the router control page because the password got changed a while ago (there have been several reboots since then so if the DHCP reservations did get changed it would have nothing to do with that).  Thanks for any help.

Link to comment

Any comment is a little pointless because of the lack of information. But the information you have been able to provide suggests that it will have to wait util you get home and then you will either sort it out quickly or post your diagnostics and receive assistance.

Link to comment
3 minutes ago, John_M said:

Any comment is a little pointless because of the lack of information. But the information you have been able to provide suggests that it will have to wait util you get home and then you will either sort it out quickly or post your diagnostics and receive assistance.

Well I can't post diagnostics if I can't access the UI, right?  And if I was at the server's location right now I wouldn't be able to do that from there either.  The only thing I'd know that I can't know from here is whether the power light is on and assuming that it is, then what?  

 

The server is hosted by my niece 100 miles from me btw because she is the one with fiber internet.  She'll be back home tonight and will tell me then if the light is on.  I'm just trying to figure out in the meantime if there's anything else I can do.  I can't be first person to run into this scenario of power is on but UI won't come up and SSH doesn't work either so I didn't think it was too much to ask that someone could offer some guidance on that without needing a log to do so.

Link to comment
12 minutes ago, trurl said:

 

Wrong.

 

If you can get to the console or telnet/ssh then you can also try to get diagnostics that way. See this "sticky" near the top of this same subforum:

 

https://lime-technology.com/forums/topic/37579-need-help-read-me-first/

 

Thanks trurl. The server is run headless and testing with my home server it seems that connecting to a monitor after it's already booted up doesn't bring up the console.  (The TV shows the input is plugged in but switching to it there is just a blank screen).  So assuming it's the same with the server out there and SSH doesn't work either, would there be any way to avoid hard powering down?  I always hate to do that though of course with the monitor then connected at boot I wouldn't have any problem accessing the console and diagnostics from there.

Link to comment
2 hours ago, ElJimador said:

So assuming it's the same with the server out there and SSH doesn't work either

I would be very very surprised if that is a valid assumption.

 

ssh is the usual way to work at the command line remotely or headless. It is possible for things to be so locked up that it won't respond but you won't know if you don't try.

Link to comment

An initial step is to verify if the machine is powered, and to also look at the indicators on the network port - is the link light lit and does the transfer light blink indicating that the machine sends/receives any network communication?

 

There are normally lots of broadcast data on the network that network cards will pick up even if you don't know what IP they have.

And if the network card hasn't found an IP, then the DHCP client in the machine should regularly try to send out broadcasts to search for a DHCP server.

Link to comment
2 hours ago, trurl said:

I would be very very surprised if that is a valid assumption.

 

ssh is the usual way to work at the command line remotely or headless. It is possible for things to be so locked up that it won't respond but you won't know if you don't try.

I have tried SSH via the control computer and it isn't working.  "Unable to open connection to JBOX. Host does not exist".  

 

Maybe something failed with the update and it shut itself off instead or booted into BIOS or something.  Until I can know I was just trying to think ahead and understand if it was in a hung state if there was anything else I'd be able to try to avoid an unclean shutdown.  Sounds like the answer is no.

Link to comment
1 hour ago, pwm said:

An initial step is to verify if the machine is powered, and to also look at the indicators on the network port - is the link light lit and does the transfer light blink indicating that the machine sends/receives any network communication?

 

There are normally lots of broadcast data on the network that network cards will pick up even if you don't know what IP they have.

And if the network card hasn't found an IP, then the DHCP client in the machine should regularly try to send out broadcasts to search for a DHCP server.

Good tips.  Thank you.

Link to comment
8 hours ago, ElJimador said:

server is run headless and testing with my home server it seems that connecting to a monitor after it's already booted up doesn't bring up the console.  (The TV shows the input is plugged in but switching to it there is just a blank screen).

This is normal as there is a builtin screen blanker. You also need to press a key on an attached keyboard to show anything.

Link to comment

Just to update, overnight my niece reported that the power light is on and remoting in to the control computer this morning I was surprised to see the main page of the UI up but with all the array details and such blank.  As you can see per the screenshot attached, it must have gotten hung up on shutdown because it's still got the notification on top about reboot required to install the update and server uptime shows 9 days+.  I'm guessing the UI reappearing was somehow triggered by an appdata backup scheduled every night to an unassigned drive that stops and restarts the dockers.  Whatever it was though, the effect of resuming a network connection at least long enough to bring up the UI didn't last because trying to SSH in still didn't work and running the IP scanner didn't show the server on the network either, while navigating to the Tools tab of the UI (hoping to bring up diagnostics) returned the familiar "This site cannot be reached" page which is all I've been able to get since.

 

Also FYI, I normally stop the array before rebooting or powering down but in this case after the update I did not.  So maybe it was the stopping the array part where it hung?  I don't know the sequencing on shutdown/reboot of when that happens vs. stopping network services and other steps.

 

Anyway I realize all this might not help much without diagnostics and I'm hoping I"ll be able to get out to the server in the next couple days and will be successful getting that via the terminal commands and can post it then.  If this update does shed any light in the meantime however please let m know.  Thanks.

 

 

JBOX_Main 5-10-18.pdf

Link to comment

Alright, got out the server and connected monitor and keyboard but could not get a terminal to come up because the monitor didn't register there was any input plugged in no matter what keys I pressed.  So I could not pull diagnostics and was forced to hard power down.  Brought the server back home and tried booting from here with all hard drives disconnected at the PSU just to be safe.  First thing I noticed was the power light came on as soon as I plugged in and flipped the switch at the PSU (without hitting the power button) but no fans came on except for CPU fan for only a split second before it quit, and repeatedly hitting delete to enter BIOS nothing happened.  Again the monitor (a different one) doesn't register any input connected.  So I try swapping the graphics card and clearing CTC RAM (first by itself then pulling the CMOS battery also)... still nothing .  Attaching a motherboard speaker gives me no error beeps at boot and the light on the ethernet port is blinking like it's trying to connect to the network but accessing the router management page and running an IP scanner from my desktop shows no server on the network.  So whatever it's doing while it's "on", I've got no graphics, no network connection, no fans, and no error indicators.

 

So I"m RMA'ing the motherboard and I guess we can consider this solved.  If there's still weirdness with the replacement board I'll post a new topic.  Thanks all.  

Link to comment
20 hours ago, pwm said:

It could also be the PSU - the NIC can operate on the 5V standby power rail (to allow wake-on-LAN) but the motherboard/processor/disks uses other rails to power up.

Really glad you suggested this.  I was all set to ship the board back to Asus today but pulling it from the case and testing with a different PSU it booted just fine.  Then pulling the old PSU to get the s/n and start the RMA for that instead I made an even better discovery: turns out one of the SATA cables I pulled from the unit was actually the modular cable to the CPU. The way the PSU is oriented in the case and with other cables in the way I just didn't see that I had pulled the wrong one.  So no wonder I couldn't get it to boot at home. 

 

Anyway, very relieved to learn there's no hardware failure after all and I'm just an idiot.  :)  It does still leave the mystery of why it got hung up on reboot in the first place but having it down for now gives me a chance to update the LSI controller firmware, review BIOS settings, and clean out all the dust bunnies, etc. now that I have to rebuild it anyway.  So I'll just cross my fingers that once I boot with the full array and rebuild parity after that there won't be any repeat.   If there are I'll revive the thread, otherwise thanks again to everyone who offered feedback.  

 

 

Link to comment

I'm having a similar issue and i think it's related to UEFI, when I reboot sometimes I just get a black screen.. I'll even power it off and turn it back on just black screen after post... but if i force a UEFI boot in the BIOS it'll come up. Never had this issue before 6.5.2 and 6.5.2-RCs

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.