thestip Posted September 12, 2021 Share Posted September 12, 2021 (edited) Hey there everyone! I could use some help figuring out why I'm getting unclean shutdowns. I've read multiple posts, checked settings, rechecked settings... Clearly, I'm missing something. Hopefully, it's not too obvious. Using NUT (had the same problem with Unraid's UPS tool, that's currently disabled) Shutdown Mode: Time on Battery Time before shutdown: 4 Current shutdown settings: VM shutdown timeout: 300 (I have 2; 1 Windows, 1 Ubuntu. The Windows box does not get automatic updates.) Disk timeout: 420 ssh,bash killed by tips and tweaks The Cyberpower UPS passed it's diag tests (their software, run on my Win10 laptop) and there's plenty of charge left on the battery after the server's off. When I manually shut down the server, it gracefully shuts down in about a minute or so. Thanks for any and all help! ------------------------------------------------- SOLUTION: New batteries. ------------------------------------------------- Edited September 28, 2021 by thestip solved Quote Link to comment
trurl Posted September 13, 2021 Share Posted September 13, 2021 Anything that prevents the "array stopped" status from being written to flash will result in unclean shutdown. That could be due to shutdown before the array is stopped, or it could be due to not being able to write the flash drive. Quote Link to comment
thestip Posted September 13, 2021 Author Share Posted September 13, 2021 (edited) I realize that, but I don't know how to go about tracking down the exact cause. I can write to the flash drive from putty, so I'm just assuming that's not the issue. How do I track down the exact cause? I read that there are logs on the flash drive when this happens, but there's no log folder on it and the only log file I see is the parity check one. Where are those logs supposed to be? What should I be looking for in them? Edited September 13, 2021 by thestip Quote Link to comment
trurl Posted September 13, 2021 Share Posted September 13, 2021 setup syslog server 1 Quote Link to comment
thestip Posted September 14, 2021 Author Share Posted September 14, 2021 Thank you. I set that up last night and the power went out again this morning. Gotta love fire season in California! I really hope I can nail down the issue later today as this happens far too often lately. Quote Link to comment
trurl Posted September 14, 2021 Share Posted September 14, 2021 1 hour ago, thestip said: power went out again Did it run on batteries long enough to get a clean shutdown? Quote Link to comment
thestip Posted September 14, 2021 Author Share Posted September 14, 2021 Nope. UPS still has 99% charge, so it has to be something in Unraid. Logs don't really tell me much... Power went off a little before 7, here's that part of the log. The full log is attached. "Sep 14 06:47:03 PlanetExpress upsmon[7634]: UPS [email protected] on battery <---- power lost here Sep 14 13:22:17 PlanetExpress kernel: microcode: microcode updated early to revision 0x1f, date = 2018-05-08" <------- power restored syslog.txt Quote Link to comment
trurl Posted September 14, 2021 Share Posted September 14, 2021 Since nothing was logged after it went on battery, that suggests you don't have NUT or UPS settings configured to shutdown. Quote Link to comment
thestip Posted September 15, 2021 Author Share Posted September 15, 2021 (edited) I switched to NUT when this started happening just to test things. Was using the normal UPS tool for years with no problem, always had clean shutdowns. If it were set to not shutdown, wouldn't it just keep running till the UPS died? Edited September 15, 2021 by thestip Quote Link to comment
thestip Posted September 15, 2021 Author Share Posted September 15, 2021 *friendly bump* 😁 Quote Link to comment
trurl Posted September 16, 2021 Share Posted September 16, 2021 If UPS software were telling it to shutdown we should have seen that in syslog. Quote Link to comment
thestip Posted September 16, 2021 Author Share Posted September 16, 2021 (edited) So any idea as to why it's not getting told to shut down or how I can troubleshoot this further? There's not a lot to configure and it looks like I have it right... Start shutdown after being on the UPS for 4 minutes. When I've come home and the power's still off, the UPS battery is still nearly full. And if it did stay on the whole time, shouldn't there be something in the syslog after it went on the UPS? Edited September 16, 2021 by thestip Quote Link to comment
ljm42 Posted September 16, 2021 Share Posted September 16, 2021 Maybe unplug the UPS while you are sitting there watching it and see what happens? Keep the syslog window open and be ready to copy data out or take a screenshot. For the first test I would stop the array first so there isn't an unclean shutdown, but you can still watch what happens. If you don't learn anything then do it again with the array started. Quote Link to comment
ChatNoir Posted September 16, 2021 Share Posted September 16, 2021 50 minutes ago, ljm42 said: Maybe unplug the UPS while you are sitting there watching it and see what happens? Keep the syslog window open and be ready to copy data out or take a screenshot. It is generally not advised to unplug as all items connected will also loose ground. It would be better to use the circuit breaker. 2 Quote Link to comment
JonathanM Posted September 16, 2021 Share Posted September 16, 2021 5 hours ago, ChatNoir said: It is generally not advised to unplug as all items connected will also loose ground. Which in total isolation is no big deal, but if any connected items still have ground, like a network switch or a monitor, things can go bang. In a very exciting and destructive fashion. If you don't have access to the breaker, temporarily plug the UPS into a switchable power strip. For bonus points plug the server in to constant power, put an equivalent load on the UPS like a small space heater, turn off the input to the UPS and see if the server shuts down properly before the dummy load loses stable power. Obviously the communication cable from the UPS stays connected to the server. Quote Link to comment
JonathanM Posted September 16, 2021 Share Posted September 16, 2021 On 9/14/2021 at 8:23 PM, thestip said: Was using the normal UPS tool for years with no problem, always had clean shutdowns. Same UPS, same batteries? SLA batteries wear out, have you done a loaded runtime test? The behavior you are describing can be perfectly explained with worn out batteries. Full surface charge, die immediately under load, still have almost full voltage when load is removed. 1 Quote Link to comment
thestip Posted September 16, 2021 Author Share Posted September 16, 2021 Great suggestions, thank you! I'll try a load test this weekend. Yes, the same UPS and batteries. I ran the self-test software with no errors, but not a load test. My bad for assuming the self test was good enough.; that's just sloppy troubleshooting. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.