Server shut down inexplicably

[spall]

Hey all,

 

I was watching a movie from my media server (unraid 6.11.5) via Kodi when the file suddenly stopped. I was getting an error from Kodi about an error reading file. Turns out my server had powered off. I have no idea why/how this happened, but what surprised me was that when I brought it back online it didn't start a parity check. Did it shut down cleanly for a mystery reason?

 

Any tips on how to figure out what happened?

 

Bonus: after resuming the film it stopped playing again and could not read the file. I expected that it had repeated itself, but the server did not shut off this time. It is possible the answer is gremlins.

 

Thanks for any help,

 

[itimpi]

25 minutes ago, spall said:

Any tips on how to figure out what happened?

 

If it did not start a parity check this almost certainly means that a shutdown command was issued at the server for some reason.   Have you eliminated any physical possibility such as someone (or a pet) momentarily pressing the power button on the server?

 

To get persistent logs that cover a reboot sequence you need to enable the Syslog Server so that if this happens again you have something that we can look at for a possible cause.

[spall]

@itimpi Yeah. 100% nobody can get to the power button. However, after looking at stuff all day I discovered buried in spam (unusual, because the email is whitelisted) I had an UPS event that triggered the shutdown. We did not lose power, and none of my other servers freaked out. So that's weird. I have noticed that one a couple power outages in the past this server seems to jumps the gun on starting a shutdown on power loss.

 

With all drives spun up the server pulls about 140W. I have it set to shutdown on 15% battery or 10 minutes of runtime. Currently it's pulling that 140W and has an estimated 56 minutes of runtime and 100% charge. Something is weird there. So I guess the mystery has changed.

 

Also, I can't explain why the movie file stopped being readable the second time when the server was still up.

 

As I have multiple servers, I really should look into setting up syslog server as you suggested. Thanks.

[JonathanM]

1 hour ago, spall said:

I have it set to shutdown on 15% battery or 10 minutes of runtime.

Why? Those settings are rather hard on the battery. Much better for the equipment to start the shutdown as soon as it's obvious the power isn't returning, in my area if the power is out more than a minute or so, it's probably going to be out for much longer.

 

I'd recommend starting the shutdown 5 to 10 minutes after power is gone at most. That way if power does come back on relatively soon, you have enough battery left for another shutdown if the returning power wasn't stable.

 

Keep in mind that recharge rates are rather slow, typically about 10 to 20 times the time on battery. So if you run 20 minutes on battery, it's probably going to be several hours before the battery is full again.

 

SLA batteries don't like to be discharged below 50%, it reduces their usable life dramatically.

 

I'm betting your battery(s) are end of life.

 

To properly test your UPS system, you need a way to cut the power to the UPS without breaking the ground connection, so switching the power without unplugging it, a load comparable to the server, like 140W of heat or incandescent lightbulbs, and a steady power source for the server.

 

Leave the communication from the UPS to the server connected, turn off the power to the UPS, and observe the UPS connected dummy load and the shutdown behaviour of the server. Hopefully the server shuts down properly before the UPS dies. Obviously the server needs power from a steady source not associated with the UPS or it's switched circuit.

[spall]

@JonathanM Yeah, the settings aren't ideal. I had actually forgot about this issue (the server shutting down immediately). I had been dialing those settings lower and lower to try and test the issue and I must have left them at that when I got sidetracked some months ago.

 

The battery could be EOL, but the unit (and battery) are less than a year old. My other servers each on their own same model UPS don't have this behavior. I should probably swap UPS and see if the problem moves with the unit itself. I'm guessing I could get a new battery under warranty.

 

I could definitely test the method you described. It had never occurred to me to leave the data cable inline and have a dummy load on the UPS.

 

Thank you, but yes, I don't normally have it set like this. My other two servers are on 10 minute timers. We tend to have a few ~5m outages every winter. I normally beat the servers to shutting down as I'm good about shutting off every computer in the house in an outage manually. This is actually how I discovered the instant shutdown issue on this box because I ran into an issue trying to shut the server down and it automated doing it simultaneously.