UPS not shutting down unRAID cleanly

[sonofdbn]

I'm running my unRAID box from an APC UPS (Back-UPS CS650), connected via USB and the server is set to shutdown after running off the UPS after the battery level reaches 25% or there are 10 minutes left. When the power trips, usually from lightning, it usually works as I would expect: everything keeps running off the UPS until power is restored, or else after a while it shuts down cleanly.

 

But occasionally, like today, it doesn't shutdown properly, and the behaviour seems odd. Firstly, there was some lightning and some of the lights went off. I reset some fuses (no more than 5 minutes) and then went to my PC (which runs off another UPS) which was in sleep mode and resumed with no problem, and went to the unRAID GUI. The Main page showed that the array was stopped and there was a message saying that there had been an unclean shutdown and the start button suggested that I should run a parity check, which I'm now doing... only another 20 hours to go....

 

• Why would there be an unclean shutdown when I'm sure there was more than enough battery capacity for the few minutes when there was no power? Can dockers or VMs prevent a clean shutdown?
• If the UPS battery did run out before a clean shutdown, shouldn't the unRAID box just be powered off and not reboot itself?

[Frank1940]

The problem with UPS is that the user often simplifies the problem.  Most will assume that the power just goes out once and stay out. Unfortunately, that is not the only way things can happen.  The power can drop out several times for various periods of time all of which are shorter than your 'trip-wire' condition.  Then when it does go out completely, the battery doesn't have enough remaining capacity to meet the 'trip-wire' and still have the reserves to safely shut the server.  (I have my Server set at 30 seconds of power outage before shutdown is initiated.  I have found if it out longer than that, it will be out for an hour or two at the absolute minimum!)

 

The next problem is that you have the condition where the UPS has been in service for several years.  The monitoring circuit thinks that battery will run for a given period of time. (That is the 'Time' you see on the setup screen in the GUI.)  But it is now old and tired.  (Think of your ten year car battery on a cold winter morning!)  It either (A) does have the reserves  it had when new and is runs down at a very fast rate before the shutdown is complete or (B) as soon as the power surge (caused when all of the servers hard drives spin up) drops the battery voltage drops low enough (due internal battery resistance) that the UPS circuitry shuts the UPS off instantaneously (thus protecting the UPS's electronics from damage). 

 

The issue with the auto restart could be the power resume settings in your BIOS.  Many MB's (server MB's in particular) have a setting to auto restart the server on power restoration.  Another issue can be that you are losing the network connection and one of your Dockers or VM's is hanging because it is trying to use that connection.   There is suppose to be a kill timer in the shut down routine but it takes much, much longer than a normal shutdown leading to a UPS battery initiated shutdown first.

[sonofdbn]

Thanks, that makes sense. Although I don't think some of the possibilities apply to my particular case (the battery is supposedly about one year old), it's clear that LOTS of things might go wrong. I think shutting down as soon quickly is a good approach. That's less inconvenient than running through a parity check (or being tempted not to).

[SSD]

Shutting down the array cleanly requires spinning up all of the drives, a power hungry operation. Your UPS's remaining time calculation is not taking this into account.

 

I had a poor experience with a UPS being expensive to maintain and not being very useful. Our power is very dependable, and we lose it maybe once a year. So the UPS only jumped into action 2 or 3 times before it wanted a new expensive battery. And, like you, I had mysterious dirty shutdowns, even after some significant testing, with no good way to figure out why. So the UPS saved me a dirty shutdown only once. I decided to not replace the batteries and live with the occasional dirty shutdown, which has never been more than a minor annoyance. I do use a quality surge protector. For those with frequent outages, a UPS makes more sense than in my situation.