Hey there everyone! I could use some help figuring out why I'm getting unclean shutdowns. I've read multiple posts, checked settings, rechecked settings... Clearly, I'm missing something. Hopefully, it's not too obvious.

 

Using NUT (had the same problem with Unraid's UPS tool, that's currently disabled)

Shutdown Mode: Time on Battery

Time before shutdown: 4

 

Current shutdown settings:

VM shutdown timeout: 300 (I have 2; 1 Windows, 1 Ubuntu. The Windows box does not get automatic updates.)

Disk timeout: 420

ssh,bash killed by tips and tweaks

 

The Cyberpower UPS passed it's diag tests (their software, run on my Win10 laptop) and there's plenty of charge left on the battery after the server's off. When I manually shut down the server, it gracefully shuts down in about a minute or so.

 

Thanks for any and all help!

 

 

-------------------------------------------------

SOLUTION: New batteries.

-------------------------------------------------

Edited September 28, 20214 yr by thestip
solved

Anything that prevents the "array stopped" status from being written to flash will result in unclean shutdown. That could be due to shutdown before the array is stopped, or it could be due to not being able to write the flash drive.

I realize that, but I don't know how to go about tracking down the exact cause. I can write to the flash drive from putty, so I'm just assuming that's not the issue.

 

How do I track down the exact cause? I read that there are logs on the flash drive when this happens, but there's no log folder on it and the only log file I see is the parity check one. Where are those logs supposed to be? What should I be looking for in them?

Edited September 13, 20214 yr by thestip

setup syslog server

Thank you. I set that up last night and the power went out again this morning. Gotta love fire season in California! I really hope I can nail down the issue later today as this happens far too often lately.

1 hour ago, thestip said:

power went out again

Did it run on batteries long enough to get a clean shutdown?

Nope. UPS still has 99% charge, so it has to be something in Unraid. Logs don't really tell me much... Power went off a little before 7, here's that part of the log. The full log is attached.

 

"Sep 14 06:47:03 PlanetExpress upsmon[7634]: UPS [email protected] on battery     <---- power lost here
Sep 14 13:22:17 PlanetExpress kernel: microcode: microcode updated early to revision 0x1f, date = 2018-05-08" <------- power restored

syslog.txt

Since nothing was logged after it went on battery, that suggests you don't have NUT or UPS settings configured to shutdown.

 

I switched to NUT when this started happening just to test things. Was using the normal UPS tool for years with no problem, always had clean shutdowns.

 

If it were set to not shutdown, wouldn't it just keep running till the UPS died?

Edited September 15, 20214 yr by thestip

*friendly bump*  😁

If UPS software were telling it to shutdown we should have seen that in syslog.

So any idea as to why it's not getting told to shut down or how I can troubleshoot this further? There's not a lot to configure and it looks like I have it right... Start shutdown after being on the UPS for 4 minutes.

 

When I've come home and the power's still off, the UPS battery is still nearly full. And if it did stay on the whole time, shouldn't there be something in the syslog after it went on the UPS?

Edited September 16, 20214 yr by thestip

Maybe unplug the UPS while you are sitting there watching it and see what happens?  Keep the syslog window open and be ready to copy data out or take a screenshot.

 

For the first test I would stop the array first so there isn't an unclean shutdown, but you can still watch what happens. If you don't learn anything then do it again with the array started.

50 minutes ago, ljm42 said:

Maybe unplug the UPS while you are sitting there watching it and see what happens?  Keep the syslog window open and be ready to copy data out or take a screenshot.

It is generally not advised to unplug as all items connected will also loose ground.

It would be better to use the circuit breaker.

5 hours ago, ChatNoir said:

It is generally not advised to unplug as all items connected will also loose ground.

Which in total isolation is no big deal, but if any connected items still have ground, like a network switch or a monitor, things can go bang. In a very exciting and destructive fashion.

 

If you don't have access to the breaker, temporarily plug the UPS into a switchable power strip.

 

For bonus points plug the server in to constant power, put an equivalent load on the UPS like a small space heater, turn off the input to the UPS and see if the server shuts down properly before the dummy load loses stable power. Obviously the communication cable from the UPS stays connected to the server.

On 9/14/2021 at 8:23 PM, thestip said:

Was using the normal UPS tool for years with no problem, always had clean shutdowns.

Same UPS, same batteries? SLA batteries wear out, have you done a loaded runtime test? The behavior you are describing can be perfectly explained with worn out batteries. Full surface charge, die immediately under load, still have almost full voltage when load is removed.

Great suggestions, thank you! I'll try a load test this weekend.

 

Yes, the same UPS and batteries. I ran the self-test software with no errors, but not a load test. My bad for assuming the self test was good enough.; that's just sloppy troubleshooting.