[Plugin] NUT v2 - Network UPS Tools

[Rysz]

Just now, Revan335 said:

Can you fix this?

That the false positive message not showing on the boot?

 

No, unfortunately, it's built directly into the NUT backend (which is not developed by me).

[Chunks]

I'm trying to figure out why my Unraid machine (NUT 2.8.1, APC ups) stays online during a self test, but my pfSense router (NUT 2.8.2 I think) running as "slave" is shutting itself off.

 

I don't know if the problem is Unraids side, or pfSense's side.

 

The Unraid log (master) shows the following:

Dec 19 18:05:56 ChunksUnraid upsmon[11191]: UPS [email protected]: administratively OFF or asleep
Dec 19 18:06:05 ChunksUnraid usbhid-ups[11164]: ups_status_set: seems that UPS [chunksups] is in OL+DISCHRG state now. Is it calibrating (perhaps you want to set 'onlinedischarge_calibration' option)? Note that some UPS models (e.g. CyberPower UT series) emit OL+DISCHRG when in fact offline/on-battery (perhaps you want to set 'onlinedischarge' option).
Dec 19 18:06:06 ChunksUnraid upsmon[11191]: UPS [email protected]: no longer administratively OFF or asleep

 

Syslog for my pfSense machine (slave) shows:

(reverse order, oldest bottom)
2023-12-19 18:06:05 	Notification (5) 	SHUTDOWN power-down by root:
2023-12-19 18:06:02 	Error (3) 	PHP-CGI notify_monitor.php: Message sent to <ME> OK
2023-12-19 18:06:00 	Notification (5) 	UPSMON Auto logout and shutdown proceeding
2023-12-19 18:06:00 	Critical (2) 	UPSMON Executing automatic power-fail shutdown
2023-12-19 18:06:00 	Notification (5) 	UPSMON UPS [email protected].<ME>: administratively OFF or asleep

 

Specifically, I don't know what to do about this line here:

Quote

Dec 19 18:06:05 ChunksUnraid usbhid-ups[11164]: ups_status_set: seems that UPS [chunksups] is in OL+DISCHRG state now. Is it calibrating (perhaps you want to set 'onlinedischarge_calibration' option)? Note that some UPS models (e.g. CyberPower UT series) emit OL+DISCHRG when in fact offline/on-battery (perhaps you want to set 'onlinedischarge' option).

How do I set this "onlinedischarge_calibration" option? I looked into the config area of NUT on Unraid, but honestly, I'm a bit uncertain how to set it correctly. Is it in "/etc/nut/upsmon.conf" at the bottom of the NUT page?

 

Any advice?

[Rysz]

24 minutes ago, Chunks said:

I'm trying to figure out why my Unraid machine (NUT 2.8.1, APC ups) stays online during a self test, but my pfSense router (NUT 2.8.2 I think) running as "slave" is shutting itself off.

 

I don't know if the problem is Unraids side, or pfSense's side.

 

The Unraid log (master) shows the following:

Dec 19 18:05:56 ChunksUnraid upsmon[11191]: UPS [email protected]: administratively OFF or asleep
Dec 19 18:06:05 ChunksUnraid usbhid-ups[11164]: ups_status_set: seems that UPS [chunksups] is in OL+DISCHRG state now. Is it calibrating (perhaps you want to set 'onlinedischarge_calibration' option)? Note that some UPS models (e.g. CyberPower UT series) emit OL+DISCHRG when in fact offline/on-battery (perhaps you want to set 'onlinedischarge' option).
Dec 19 18:06:06 ChunksUnraid upsmon[11191]: UPS [email protected]: no longer administratively OFF or asleep

 

Syslog for my pfSense machine (slave) shows:

(reverse order, oldest bottom)
2023-12-19 18:06:05 	Notification (5) 	SHUTDOWN power-down by root:
2023-12-19 18:06:02 	Error (3) 	PHP-CGI notify_monitor.php: Message sent to <ME> OK
2023-12-19 18:06:00 	Notification (5) 	UPSMON Auto logout and shutdown proceeding
2023-12-19 18:06:00 	Critical (2) 	UPSMON Executing automatic power-fail shutdown
2023-12-19 18:06:00 	Notification (5) 	UPSMON UPS [email protected].<ME>: administratively OFF or asleep

 

Specifically, I don't know what to do about this line here:

How do I set this "onlinedischarge_calibration" option? I looked into the config area of NUT on Unraid, but honestly, I'm a bit uncertain how to set it correctly. Is it in "/etc/nut/upsmon.conf" at the bottom of the NUT page?

 

Any advice?

 

Hello!

 

This is caused by the pfSense side and is the fallout of the following bug:

https://github.com/networkupstools/nut/issues/2104

 

You cannot fix this from the UNRAID side, you need to update NUT on your pfSense to the latest version. The bug is already fixed in NUT 2.8.1 (or 2.8.2 on pfSense), so please check the pfSense's NUT version again because it looks like a version before 2.8.1. from the logs! After updating to the latest version on the pfSense, you can then test it by running a manual battery self-test on the UPS and checking if it still shuts down the pfSense (it shouldn't anymore). On UNRAID itself no changes are needed as it's already fixed there. 🙂

 

Edited December 19, 2023 by Rysz

[jimeez]

On 9/4/2023 at 7:44 PM, DataBitz said:

Just wanted to share my working configuration where Unraid is a Slave and pfsense is the Master, for anyone else looking.

Eaton 3S 850 connected via USB to pfsense (192.168.1.1)

 

pfsense config

 

 

Unraid NUT v2 config.

 

 

I am curious to know if you had to make any changes to this configuration recently?  I tried applying the same settings (I am also using an Eaton UPS unit [5P 1500]) but am not able to get Unraid to grab info from the pfSense gateway.  Also, I do not have a UPS Slave Username or UPS Slave Password on the Unraid settings.   This is all that is available to me.

 

 

Edited December 20, 2023 by jimeez

[Rysz]

Just now, jimeez said:

 

I am curious to know if you had to make any changes to this configuration recently?  I tried applying the same settings (I am also using an Eaton UPS unit [5P 1500]) but I do not have a UPS Slave Username or UPS Slave Password on the Unraid settings.   This is all that is available to me.

 

 

 

You need to set NUT Mode to Netserver.

[jimeez]

1 minute ago, Rysz said:

 

You need to set NUT Mode to Netserver.

On the Unraid side or pfSense side?  Just to clarify, I am using (or attempting to use) pfSense as the master and Unraid as the slave.

 

Thank you for the amazingly fast reply by the way. 

[Rysz]

9 minutes ago, jimeez said:

On the Unraid side or pfSense side?  Just to clarify, I am using (or attempting to use) pfSense as the master and Unraid as the slave.

 

Thank you for the amazingly fast reply by the way. 

 

Ah, my bad, in that case you'll put the pfSense's configured Slave username and password (that'd be monslave and yourpassword in the original screenshot) into the respective NUT Monitor username and password fields on the UNRAID server. NUT Mode would, of course, be Slave on the UNRAID server. 🙂

The second credential fields were recently hidden because they were unused in NUT Mode Slave.

That's also the reason it didn't matter the user from the screenshot had the same values in them.

 

Edited December 20, 2023 by Rysz

[jimeez]

8 minutes ago, Rysz said:

 

Ah, my bad, in that case you'll put the pfSense's configured Slave username and password (that'd be monslave and yourpassword in the original screenshot) into the respective NUT Monitor username and password fields on the UNRAID server. NUT Mode would be Slave on the UNRAID server. 🙂

The second credential fields were recently hidden because they were unused in NUT Mode Slave.

 

Yep, that is exactly what I did. 

 

  

 

However, the UNRAID server does not display any UPS information.  it just sits there and "spins".  Then evetually indicates that there "no information available".

 

  

Edited December 20, 2023 by jimeez

[Rysz]

1 minute ago, jimeez said:

Yep, that is exactly what I did. 

 

  

 

However, the UNRAID server does not display any UPS information.  it just sits there and "spins". 

 

  

 

That's weird, is there anything in the UNRAID logs indicating a problem?

Is it possible that pfSense (being a firewall) blocks LAN-access in/out from TCP Port 3493 (NUT)?

[jimeez]

5 minutes ago, Rysz said:

That's weird, is there anything in the UNRAID logs indicating a problem?

 

Well how about that.  There sure is. 

 

 

I'll do some poking around in pfSense and see if it's being blocked somehow.  Good suggestion.

Edited December 20, 2023 by jimeez

[Rysz]

Just now, jimeez said:

Well how about that.  There sure is. 

 

 

 

I'll do some poking around in pfSense and see if it's being blocked somehow.  Good suggestion.

 

Indeed it seems like the pfSense is dropping the incoming packets to your NUT server's port on TCP 3493.

Please do report back if you managed to open the port and if it worked out for you eventually. 🙂

[Rysz]

15 minutes ago, jimeez said:

Well how about that.  There sure is. 

 

 

 

I'll do some poking around in pfSense and see if it's being blocked somehow.  Good suggestion.

A quick search on Google returned another thing worth trying could be adding this on your pfSense:

 

But judging from the screenshot of the original poster this shouldn't be necessary anymore. The requirement of an open TCP port 3493 for inbound/outbound on the LAN remains, so I think that's the main issue here.

 

Edited December 20, 2023 by Rysz

[jimeez]

21 minutes ago, Rysz said:

 

Indeed it seems like the pfSense is dropping the incoming packets to your NUT server's port on TCP 3493.

Please do report back if you managed to open the port and if it worked out for you eventually. 🙂

 

I figured it out by using some of the settings in this write-up.   I had to add additional configurations to ups.conf and upsd.conf.  The instant I hit SAVE on the pfSense box, UNRAID saw the info instantly.

 

EDIT: the ups.conf line is NOT needed.  Only upsd.conf setting.

 

 

 

One last remaining question if I may.  And understanding that this is a support thread for the UNRAID plugin, but figured it's worth asking.  How can I adjust the setting on the pfSense side so that my pfSense device does not shut down until the very last minutes of remaining battery life? 

 

Many thanks for the support today.

Edited December 20, 2023 by jimeez

[Rysz]

2 minutes ago, jimeez said:

 

I figured it out by using some of the settings in this write-up.   I had to add additional configurations to ups.conf and upsd.conf.  The instant I hit SAVE on the pfSense box, UNRAID saw the info instantly.

 

 

 

One last remaining question if I may.  And understanding that this is a support thread for the UNRAID plugin, but figured it's worth asking.  How can I adjust the setting on the pfSense side so that my pfSense device does not shut down until the very last minutes of remaining battery life? 

 

Many thanks for the support today.

 

pfSense NUT and NUT in general will by default shutdown when the UPS sends the low battery signal "LOWBATT" (usually happens around 20% charge). The runtime remaining or battery percentage remaining thresholds which trigger the low battery condition are often configurable on the UPS itself. I also happen to have an Eaton 5P and you can configure this on the UPS screen here:

So UNRAID also always does this as a last resort (shutting down on a low battery signal), but also has other additional shutdown conditions which act on top of that to ensure that UNRAID has sufficient time to shutdown before any such low battery signal. So setting a reasonable time for the additional shutdown condition (e.g. Time on Battery) on UNRAID while keeping the NUT defaults on the pfSense would ensure that UNRAID shuts down before the pfSense (and any other devices using basic NUT).

[jimeez]

19 minutes ago, Rysz said:

 

pfSense NUT and NUT in general will by default shutdown when the UPS sends the low battery signal "LOWBATT" (usually happens around 20% charge). The runtime remaining or battery percentage remaining thresholds which trigger the low battery condition are often configurable on the UPS itself. I also happen to have an Eaton 5P and you can configure this on the UPS screen here:

So UNRAID also always does this as a last resort (shutting down on a low battery signal), but also has other additional shutdown conditions which act on top of that to ensure that UNRAID has sufficient time to shutdown before any such low battery signal. So setting a reasonable time for the additional shutdown condition (e.g. Time on Battery) on UNRAID while keeping the NUT defaults on the pfSense would ensure that UNRAID shuts down before the pfSense (and any other devices using basic NUT).

 

Perfect.  Thank you so much.  This is THE best explanation I have read so far.  Makes so much sense.

 

Thank you again for your help.  Hopefully others benefit from this.

[RAINMAN]

Can anyone help me figure out why this stopped working?  At some point I updated it and it hasnt worked since.

 

It seems there is a fatal error then the driver doesnt work:

 

Quote

Dec 23 11:07:51 FILESERVER root: Writing NUT configuration...
Dec 23 11:07:52 FILESERVER root: Updating permissions for NUT...
Dec 23 11:07:52 FILESERVER root: Checking if the NUT Runtime Statistics Module should be enabled...
Dec 23 11:07:52 FILESERVER root: Disabling the NUT Runtime Statistics Module...
Dec 23 11:07:53 FILESERVER root: Network UPS Tools - BCMXCP UPS driver 0.33 (2.8.1)
Dec 23 11:07:53 FILESERVER root: USB communication subdriver 0.27
Dec 23 11:07:53 FILESERVER root: 
Dec 23 11:07:53 FILESERVER root: Fatal error: 'bus' is not a valid variable name for this driver.
Dec 23 11:07:53 FILESERVER root: 
Dec 23 11:07:53 FILESERVER root: Look in the man page or call this driver with -h for a list of
Dec 23 11:07:53 FILESERVER root: valid variable names and flags.
Dec 23 11:07:53 FILESERVER root: Network UPS Tools - UPS driver controller 2.8.1
Dec 23 11:07:55 FILESERVER upsd[6971]: listening on 0.0.0.0 port 3493
Dec 23 11:07:55 FILESERVER upsd[6971]: Can't connect to UPS [ups] (bcmxcp_usb-ups): No such file or directory
Dec 23 11:07:55 FILESERVER upsd[6971]: Found 1 UPS defined in ups.conf
Dec 23 11:07:55 FILESERVER upsd[6972]: Startup successful
Dec 23 11:07:55 FILESERVER upsmon[6975]: Startup successful
Dec 23 11:07:55 FILESERVER upsmon[6976]: [D1:6976] Saving PID 6976 into /var/run/nut/upsmon.pid
Dec 23 11:07:55 FILESERVER upsmon[6976]: [D1:6976] Succeeded to become_user(root): now UID=0 GID=0
Dec 23 11:07:55 FILESERVER upsmon[6976]: [D1:6976] upsnotify: failed to notify about state 2: no notification tech defined, will not spam more about it
Dec 23 11:07:55 FILESERVER upsmon[6976]: [D1:6976] Trying to connect to UPS [[email protected]]
Dec 23 11:07:55 FILESERVER upsd[6972]: User [email protected] logged into UPS [ups]
Dec 23 11:07:55 FILESERVER upsmon[6976]: [D1:6976] Logged into UPS [email protected]
Dec 23 11:07:55 FILESERVER upsmon[6976]: Poll UPS [[email protected]] failed - Driver not connected
Dec 23 11:07:55 FILESERVER upsmon[6976]: Communications with UPS [email protected] lost
Dec 23 11:08:00 FILESERVER upsmon[6976]: Poll UPS [[email protected]] failed - Driver not connected
Dec 23 11:08:00 FILESERVER upsmon[6976]: UPS [email protected] is unavailable
Dec 23 11:08:05 FILESERVER upsmon[6976]: Poll UPS [[email protected]] failed - Driver not connected
Dec 23 11:08:10 FILESERVER upsmon[6976]: Poll UPS [[email protected]] failed - Driver not connected

 

 

I've tried different USB ports, rebooting, cant seem to get it working again.

[Rysz]

2 minutes ago, RAINMAN said:

Can anyone help me figure out why this stopped working?  At some point I updated it and it hasnt worked since.

 

It seems there is a fatal error then the driver doesnt work:

 

 

 

I've tried different USB ports, rebooting, cant seem to get it working again.

I'd try to do "Reset Config" in NUT Settings and start over with a fresh NUT configuration, it seems your configuration file includes a custom "bus" variable that's not supported by this driver. What kind of UPS do you have that's using this exotic UPS driver? 

[RAINMAN]

4 minutes ago, Rysz said:

I'd try to do "Reset Config" in NUT Settings and start over with a fresh NUT configuration, it seems your configuration file includes a custom "bus" variable that's not supported by this driver. What kind of UPS do you have that's using this exotic UPS driver? 

 

Its a powerware 9120 (oldie but works well)

 

When I did that I get:

 

Quote

Dec 23 11:17:59 FILESERVER root: Unable to find POWERWARE UPS device on USB bus (USB)
Dec 23 11:17:59 FILESERVER root: 
Dec 23 11:17:59 FILESERVER root: Things to try:
Dec 23 11:17:59 FILESERVER root: 
Dec 23 11:17:59 FILESERVER root:  - Connect UPS device to USB bus
Dec 23 11:17:59 FILESERVER root: 
Dec 23 11:17:59 FILESERVER root:  - Run this driver as another user (upsdrvctl -u or 'user=...' in ups.conf).
Dec 23 11:17:59 FILESERVER root:    See upsdrvctl(8) and ups.conf(5).
Dec 23 11:17:59 FILESERVER root: 
Dec 23 11:17:59 FILESERVER root: Driver failed to start (exit status=1)

 

When I unplugged the USB and put it into another slot then tried to start it, it came up.  Excellent!  Not sure why its so finicky sometimes but appreciate the help getting it working.

 

Minor issue:  By syslog is getting spammed with the following.  Unsure if it matters or how to turn that off.

 

Quote

Dec 23 11:21:27 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: OUTGOING_MODEM_CALL_STARTED
Dec 23 11:21:29 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: STARTUP_FAILURE_CHECK_EPO
Dec 23 11:21:29 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: OUTGOING_MODEM_CALL_STARTED
Dec 23 11:21:31 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: STARTUP_FAILURE_CHECK_EPO
Dec 23 11:21:31 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: OUTGOING_MODEM_CALL_STARTED
Dec 23 11:21:33 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: STARTUP_FAILURE_CHECK_EPO
Dec 23 11:21:33 FILESERVER bcmxcp_usb[11063]: alarm_set: result was truncated while setting or appending alarm_buf (limited to 256 bytes), with message: OUTGOING_MODEM_CALL_STARTED

 

[Rysz]

23 minutes ago, RAINMAN said:

 

Its a powerware 9120 (oldie but works well)

 

When I did that I get:

 

 

When I unplugged the USB and put it into another slot then tried to start it, it came up.  Excellent!  Not sure why its so finicky sometimes but appreciate the help getting it working.

 

Minor issue:  By syslog is getting spammed with the following.  Unsure if it matters or how to turn that off.

 

 

 

Glad you got it working again, changing USB ports can sometimes work wonders with driver-device recognition. I see that it's listed as an experimental driver in the NUT manuals and I'm guessing that since it's a relatively exotic one at that there's not much active development going on there.

 

The log messages are caused by the driver, unfortunately, and I'll file this with the NUT backend developers but right now there's nothing we can do about that within NUT (might be possible configuring your syslog to discard these messages somehow). These likely won't cause any harm or impair the function of NUT if the important UPS data itself shows normally within the NUT dashboards (particularly the UPS Status as Online). 

 

The messages mostly seem caused by the UPS emitting (longer) non-generic, additional status messages that the UPS driver isn't able to handle or understand (yet), likely because right now it's only programmed for the most important states (e.g. UPS Status, UPS Battery Charge etc.) to ensure basic NUT functionality. But you can see here what those additional status messages mean on your UPS: https://networkupstools.org/protocols/eaton/XCP_Alarm_Map_021309.pdf

 

I wouldn't worry about it too much overall, but might be worth doing a blackout simulation to see if everything works as it should before relying on the UPS in production.

 

Edited December 23, 2023 by Rysz

[RAINMAN]

53 minutes ago, Rysz said:

 

Glad you got it working again, changing USB ports can sometimes work wonders with driver-device recognition. I see that it's listed as an experimental driver in the NUT manuals and I'm guessing that since it's a relatively exotic one at that there's not much active development going on there.

 

The log messages are caused by the driver, unfortunately, and I'll file this with the NUT backend developers but right now there's nothing we can do about that within NUT (might be possible configuring your syslog to discard these messages somehow). These likely won't cause any harm or impair the function of NUT if the important UPS data itself shows normally within the NUT dashboards (particularly the UPS Status as Online). 

 

The messages mostly seem caused by the UPS emitting (longer) non-generic, additional status messages that the UPS driver isn't able to handle or understand (yet), likely because right now it's only programmed for the most important states (e.g. UPS Status, UPS Battery Charge etc.) to ensure basic NUT functionality. But you can see here what those additional status messages mean on your UPS: https://networkupstools.org/protocols/eaton/XCP_Alarm_Map_021309.pdf

 

I wouldn't worry about it too much overall, but might be worth doing a blackout simulation to see if everything works as it should before relying on the UPS in production.

 

 

Thanks.  Its odd because I used the old NUT plugin for years and never saw those messages.  This new plugin worked briefly no messages then randomly stopped and now its working again but its kind of unusable as my syslog is filling up.  I cant seem to find a way to turn off messages.  Any way I can totally disable all syslog witting from NUT?  The plugin is running fine and its not in any critical environment so as long as it gives time to do a proper shutdown when I lose power thats all I need.

 

This is what is getting truncated:

 

Quote

SOFTWARE_INCOMPABILITY_DETECTED IN_PARALLEL_OPERATION CLOSE_BYPASS_BREAKER AUTO_ON_COMMAND_EXECUTED ON_DOUBLE_BOOST UPS_CABINET_OVER_TEMP TRANSFORMER_OVER_TEMP AMBIENT_UNDER_TEMP AMBIENT_OVER_TEMP CABINET_DOOR_OPEN_VOLT_PRESENT AUTO_SHUTDOWN_PENDING STAR

 

Not sure what all that means as the unit isnt displaying any noticeable alarms and most wouldn't even apply.  Maybe those are possible alarms. unsure.

[Rysz]

22 minutes ago, RAINMAN said:

 

Thanks.  Its odd because I used the old NUT plugin for years and never saw those messages.  This new plugin worked briefly no messages then randomly stopped and now its working again but its kind of unusable as my syslog is filling up.  I cant seem to find a way to turn off messages.  Any way I can totally disable all syslog witting from NUT?  The plugin is running fine and its not in any critical environment so as long as it gives time to do a proper shutdown when I lose power thats all I need.

 

This is what is getting truncated:

 

 

Not sure what all that means as the unit isnt displaying any noticeable alarms and most wouldn't even apply.  Maybe those are possible alarms. unsure.

 

It might be changes to the overall driver structure or underlying state handling that causes this in newer NUT backend versions. What you can do, if the old plugin worked for you without such problems, is use the "NUT Backend Switch" in the NUT Settings to change the NUT backend to "legacy (2.7.4)".

 

After a restart (or NUT reinstall, if you can't restart) you'll then be using the exact same (older) backend as the previous plugin did and hopefully that'll resolve your log situation once again.

 

Do make sure "NUT Backend" in fact changes from 2.8.1 to 2.7.4 to ensure the backend was in fact changed successfully. Please let me know if that worked for you.

 

Edited December 23, 2023 by Rysz

[RAINMAN]

4 hours ago, Rysz said:

 

It might be changes to the overall driver structure or underlying state handling that causes this in newer NUT backend versions. What you can do, if the old plugin worked for you without such problems, is use the "NUT Backend Switch" in the NUT Settings to change the NUT backend to "legacy (2.7.4)".

 

After a restart (or NUT reinstall, if you can't restart) you'll then be using the exact same (older) backend as the previous plugin did and hopefully that'll resolve your log situation once again.

 

Do make sure "NUT Backend" in fact changes from 2.8.1 to 2.7.4 to ensure the backend was in fact changed successfully. Please let me know if that worked for you.

 

 

This worked.  Thank you!  Your help today was much appreciated.  

 

No syslog spamming on v2.7.4

 

Quote

Dec 23 17:31:37 FILESERVER root: Network UPS Tools - UPS driver controller 2.7.4.1
Dec 23 17:31:38 FILESERVER upsd[14461]: listening on 0.0.0.0 port 3493
Dec 23 17:31:38 FILESERVER upsd[14461]: Connected to UPS [ups]: bcmxcp_usb-ups
Dec 23 17:31:38 FILESERVER upsd[14462]: Startup successful
Dec 23 17:31:38 FILESERVER upsmon[14465]: Startup successful
Dec 23 17:31:38 FILESERVER upsd[14462]: User [email protected] logged into UPS [ups]

 

[scania471]

Hi!

 

I have been having a random disconnect problem for the last year and I can't seem to figure out why it happens.

I have an APC Back-UPS BX950MI and it randomly disconnects from the server. I have to attach it to a VM, then detach it so the system recognizes it.

Previously, the boot USB got disconnected as well and could not remount, only after reboot. That time I thought the USB controller was faulty on the motherboard since I tried every port configuration. Weirdly enough, an external HDD connected to the server did not disconnect. This happened with 6.9.2 and 6.10.3. Since then, I upgraded the license to Plus, updated to 6.12.4, replaced the pendrive and it only disconnects the UPS, every other USB device is fine.

 

If I look at the logs, it starts spamming the following when the UPS is not detected:

Dec 31 01:56:26 New-Whonnock upsmon[5327]: Poll UPS [[email protected]] failed - Data stale
Dec 31 01:56:28 New-Whonnock usbhid-ups[5319]: device->Product is NULL so it is not possible to determine whether to activate max_report_size workaround
Dec 31 01:56:28 New-Whonnock usbhid-ups[5319]: libusb1: Could not open any HID devices: insufficient permissions on everything
Dec 31 01:56:30 New-Whonnock usbhid-ups[5319]: device->Product is NULL so it is not possible to determine whether to activate max_report_size workaround
Dec 31 01:56:30 New-Whonnock usbhid-ups[5319]: libusb1: Could not open any HID devices: insufficient permissions on everything

 

Tried a different cable and it stopped doing it for a month but then it started doing it again.

It usually happens every 2-3 weeks, most of the time between 1-3am when the maintenance tasks are running (docker backup, plex, vm backups, etc...) but I also experienced a disconnect with the previous boot drive that I was doing a parity rebuild during the day and it disconnected the boot drive. It did it 2x in a row, so I had to do the rebuild again and again. It looks like it does it when under some kind of load or something.

 

I attached the full log file and the configuration page of the UPS. Today, the last consumption info came in at 1:14am according to my Grafana dashboard.

 

Thank you very much in advance for your help!

syslog.1.txt

[Rysz]

On 12/31/2023 at 12:11 PM, scania471 said:

Hi!

 

I have been having a random disconnect problem for the last year and I can't seem to figure out why it happens.

I have an APC Back-UPS BX950MI and it randomly disconnects from the server. I have to attach it to a VM, then detach it so the system recognizes it.

Previously, the boot USB got disconnected as well and could not remount, only after reboot. That time I thought the USB controller was faulty on the motherboard since I tried every port configuration. Weirdly enough, an external HDD connected to the server did not disconnect. This happened with 6.9.2 and 6.10.3. Since then, I upgraded the license to Plus, updated to 6.12.4, replaced the pendrive and it only disconnects the UPS, every other USB device is fine.

 

If I look at the logs, it starts spamming the following when the UPS is not detected:

Dec 31 01:56:26 New-Whonnock upsmon[5327]: Poll UPS [[email protected]] failed - Data stale
Dec 31 01:56:28 New-Whonnock usbhid-ups[5319]: device->Product is NULL so it is not possible to determine whether to activate max_report_size workaround
Dec 31 01:56:28 New-Whonnock usbhid-ups[5319]: libusb1: Could not open any HID devices: insufficient permissions on everything
Dec 31 01:56:30 New-Whonnock usbhid-ups[5319]: device->Product is NULL so it is not possible to determine whether to activate max_report_size workaround
Dec 31 01:56:30 New-Whonnock usbhid-ups[5319]: libusb1: Could not open any HID devices: insufficient permissions on everything

 

Tried a different cable and it stopped doing it for a month but then it started doing it again.

It usually happens every 2-3 weeks, most of the time between 1-3am when the maintenance tasks are running (docker backup, plex, vm backups, etc...) but I also experienced a disconnect with the previous boot drive that I was doing a parity rebuild during the day and it disconnected the boot drive. It did it 2x in a row, so I had to do the rebuild again and again. It looks like it does it when under some kind of load or something.

 

I attached the full log file and the configuration page of the UPS. Today, the last consumption info came in at 1:14am according to my Grafana dashboard.

 

Thank you very much in advance for your help!

syslog.1.txt 1.95 MB · 0 downloads

 

If you are using "powertop", this is known to cause instabilities with some UPS (changing the auto-hibernation of USB devices). If not, continue reading...

 

USB connectivity problems (data staleness, timeouts, ...) are incredibly hard if not impossible to diagnose remotely. They can stem from a multitude of factors because of the many parts involved in the equation (USB cable, port, bus, driver, implementation, ...). Common reasons have included broken USB cables (could attempt new USB cable), USB ports (could attempt a different USB port), USB buses (either on the host system or the UPS itself) and/or overall instability of the UPS driver in combination with a specific USB bus (could attempt PCIe USB card, could try on another PC).

 

What's also proven to resolve the problem for some is using the serial cable or SNMP network connection instead of USB (where possible and available) for more overall stability and/or playing around with the different available NUT backends (utilizing the "NUT Backend Switch" in "NUT Settings") to resolve any possible driver and/or bus instabilities.

 

See also here: https://networkupstools.org/docs/man/usbhid-ups.html (section "KNOWN ISSUES AND BUGS" regarding "Repetitive timeout and staleness") for the configurational values "pollinterval" and "pollfreq". Please do consult the NUT user manual on their scope and where to apply them in configuration files, when doing so please be advised to put them in the marked safe zones in the respective configuration files (best utilizing the "NUT Configuration Editor" in "NUT Settings") for them not to get overwritten.

 

https://networkupstools.org/docs/user-manual.chunked/index.html

 

Edited February 27 by Rysz

[user-115]

On 12/2/2023 at 1:37 PM, user-115 said:

To anybody who finds this thread.

 

As of 12/02/2023, NUT is NOT compatible with UnRaid v6.12.6, and it will be automatically removed upon upgrading.

When this compatibility issue is fixed, please reply to my comment saying it's fixed.

Just wanted to get this on here in case anyone is thinking of upgrading UnRaid, and wants to keep NUT.

At this point, NUT is working again.
However, it's worth noting that I had followed the instructions for manual installation beforehand.
This time around, it was available in the Apps section, and now works well. Better than before, because now NUT comes up on my dashboard, instead of just the bottom of the window.

Definitely my go to for UPS management.