UPS not working correctly


Recommended Posts

So I'm using a CyberPower CP1500PFCLCD as a UPS for my UnRAID server.  Lately, I've had two power failures that resulted in unclean shutdowns and a subsequent parity check upon reboot.  I can't figure out how to see the syslog for before the shutdown occurred?  Is this available somewhere as I checked my log folder on my flash drive and I don't see anything?  My guess is the server shutdown immediately on power loss but no way to confirm without manually pulling the plug to the UPS (which I'd like to avoid  if at all possible).

 

After the first failure, I changed my UPS settings to initiate a powerdown after 30 seconds.  Previously, I had it set to shutdown with 10% power remaining.  The UPS is relatively new so it's not like the battery is bad.  My dashboard shows the UPS is connected and I've attached a screenshot of my settings page.  Feels like something isn't configured correctly?  Any suggestions appreciated.

battery.JPG

Edited by betaman
Link to comment
5 minutes ago, bjp999 said:

You could stop the array and then pull the plug on the UPS. You'd not risk a dirty shutdown and could watch what happens.

Even better, temporarily plug the server directly into just a surge protector that will stay on, and plug a couple 100W lamps into the UPS as a test load. Turn off the power to the UPS, and make sure the server shuts down cleanly before the light bulbs go out. Some people say it's ok to pull the plug on the UPS to test it, but after I blew up some equipment because the UPS was no longer grounded, I now put the UPS on an easily switched circuit before testing so the ground path stays connected the whole time.

Link to comment

Giving more thought - I might suggest stopping the array with all drives spinning, cut power to the UPS (safely as @jonathanm mentions), and let it sit there until the server dies. That would tell you how much run-time you have with the UPS with all drives spinning.

 

Give UPS plenty of time to recharge.

 

I would run all of the dockers and VMs you normally run. Then I would spin down all of the disks (with array started), cut power to the UPS, and manually stop the sever, and then let it run until the server dies. Measure amount of time to stop the array, and then amount of time after the array is stopped and before the server dies. If you can, determine the % of power left in the UPS after the stop array stops.

 

Those data points would help in troubleshooting. At the end of this, you should have a good sense of the practical limits of the UPS power.

 

Assuming you are satisfied the UPS has enough power, you could test with zero delay. Cut the power, and watch the UPS try to powerdown the server. If it isn't happening, or taking an long time, you can always re-engage the power to the UPS, and forestall a dirty shutdown. You should have a pretty good sense of how much time you have based on the experiments above.

 

Experiment further. But remember, the batteries loose power over time. I would be very conservative to give the server plenty of time to shut down.

Link to comment

Thanks for the suggestions guys.  Hard to imagine the battery doesn't have enough juice for 30 seconds of runtime plus shutdown.  One thing I've noticed that appears to prolong shutdown is the stopping of my Win 7 VM.  Whenever I manually restart the server for updates, I make it a habit of shutting down the VM from within Windows (i.e. use shutdown command from within Windows).  That seems to dramatically improve the total time to stop the array and shutdown the server.  In either case though, we're talking about a min or two difference so 1-2 vs. 3-4 min total.

 

I take it there's no way to confirm the server actually saw the loss of power before shutdown then?  I was hoping a log file might still be available but I'm guessing this is a limitation of the log file running in RAM when the shutdown occurs?

 

I'll try the procedure outlined above and report back later.

Edited by betaman
Link to comment
25 minutes ago, betaman said:

One thing I've noticed that appears to prolong shutdown is the stopping of my Win 7 VM.

Install apcupsd in the VM, and point it at the server. You can tell the VM to start its own shut down process before the server starts its shut down. That way by the time the server is shutting down, the VM has already halted.

Link to comment

I believe the shutdown configuration calls the powerdown script, and the powerdown script saves the syslog before shutdown (probably after the array stops). So if it never shuts down and loses power, it might never happen. I think it puts the file in the /logs directory on the flash.

 

About having enough power, the thing says it will give you a minimum of 2 minutes. I tend to think of the min as the likely :(. Doesn't seem overly generous, but your testing will determine. That 11 minutes is a pipe dream I expect! 

 

You need to do the time trials I laid out to really be in a position to really be able to speak to how much time you have.

 

But I could see powerdown getting hung up on something, and the server dying on the vine because it doesn't finish its thing. 

Link to comment
3 minutes ago, bjp999 said:

I believe the shutdown configuration calls the powerdown script, and the powerdown script saves the syslog before shutdown (probably after the array stops). So if it never shuts down and loses power, it might never happen. I think it puts the file in the /logs directory on the flash.

 

It's weird because my logs directory on the flash seems to have stopped saving logs to this location around early February.  I don't remember configuring a different location for the log files? Is that even possible?

Link to comment
7 minutes ago, jonathanm said:

Install apcupsd in the VM, and point it at the server. You can tell the VM to start its own shut down process before the server starts its shut down. That way by the time the server is shutting down, the VM has already halted.

 

Nice idea...once I get the first apcupsd issue resolved I'll try adding this to the mix!

Link to comment

I looked at your UPS settings page (in your first post) and noticed that the "UPS Load and UPS Load %  readings were both zero.  Are you sure that you have pluged the server into a battery protected outlet on the UPS?  (I know this is like a customer service representative asking if a dead computer is plugged into the wall outlet but there is always a reason to this sort of question!)

Edited by Frank1940
  • Upvote 2
Link to comment
12 minutes ago, Frank1940 said:

I looked at your UPS settings page (in your first post) and noticed that the "UPS Load and UPS Load %  readings were both zero.

Good catch. Hard for the UPS to calculate things properly if the load isn't being sensed. Maybe a defective UPS?

(or you were right in your first guess about the server not being on a protected circuit)

 

The dummy load lightbulb test is looking more and more valuable here if things are plugged in correctly. (not incorrectly)xD

 

  • Upvote 1
Link to comment
3 hours ago, Frank1940 said:

I looked at your UPS settings page (in your first post) and noticed that the "UPS Load and UPS Load %  readings were both zero.  Are you sure that you have pluged the server into a battery protected outlet on the UPS?  (I know this is like a customer service representative asking if a dead computer is plugged into the wall outlet but there is always a reason to this sort of question!)

 

Yep...that would be the problem! Nice catch. I can't even venture a guess as to when/how that hapepened! Nevermind the obvious cues from the dashboard (missing load info and 175.5 min of runtime!).  Anyway, thanks for endulging my stupidity!

Link to comment

Well that would certainly explain a lot.

 

Still think would be worthwhile to do some of the testing I indicated to help get your settings optimized.

 

(And for the record, the very first test I suggest would have found the problem, and done no harm.)

 

Good luck!

Link to comment
15 hours ago, bjp999 said:

Well that would certainly explain a lot.

Still think would be worthwhile to do some of the testing I indicated to help get your settings optimized.

(And for the record, the very first test I suggest would have found the problem, and done no harm.)

Good luck!

 

Yeah, I don't disagree.  I also need to get my VM configured to shutdown prior to the server shutdown.  Thanks again.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.