[6.9.0 betas & 6.9.1] Apcupsd issue when booting

I'm experiencing the same issue with a SMC1000I-2UC, if that cable fixes it I'll get one ordered.

23 hours ago, sturgeo said:

I'm experiencing the same issue with a SMC1000I-2UC, if that cable fixes it I'll get one ordered.

Thanks for replying, that's reassuring to see I'm not alone -at all- with this long lasting issue, as we have exactly the same UPS, except for the form factor (rack mount for you, tower for me).

I should receive the serial cable on Tuesday, and I'll share my findings, of course.

I read apcupsd manual again, and the setup should be rather straightforward according to http://apcupsd.org/manual/manual.html#modbus-driver .

As i'm not a Linux expert, and haven't played with serial ports for years if not decades, I just wonder how to find the device name. I have an existing /dev/ttyS0 entry, I suppose it is the DB9 serial port of my motherboard, but not 100% sure.

I'll keep you informed anyway.

Hi all,

I just received the serial cable and have a first positive feedback.

I stopped the apcupsd daemon, unplugged the USB cable, plugged the serial cable. Then I modified the UPS settings as per the apcupsd manual : 

 

 

 

After restarting the apcupsd daemon, and a few seconds for refreshing, the daemon obviously gets consistent data from the UPS :

/dev/ttyS0 was obviously the unique DB9 serial port on my motherboard, and all values are correct.

I have simulated a short power outage of 2 mins (without shutdown, batteries @ 90% and 18 mins runtime left), the daemon got the expected data and had the expected behavior :

 

I still have to reboot the server to make sure everything starts over properly after a reboot (which was the problem with the USB cable), but I can't reboot right now, it's already a bit late here.

If everything is OK, I'll simulate a power outage long enough to trigger the shutdown, and check what happens when mains returns.

 

I hope I can test all that tomorrow morning and will report back.

Sound good this solve the problem.

 

Interesting part was it work with modbus over serial. No parameters such as buad rate, stop bit etc ......

 

When I use serial ( not support mobus ), due to low buad rate, to transfer one report it use more then 1 sec and sometimes will corrupt.

2 hours ago, Vr2Io said:

Sound good this solve the problem.

 

Interesting part was it work with modbus over serial. No parameters such as buad rate, stop bit etc ......

 

When I use serial ( not support mobus ), due to low buad rate, to transfer one report it use more then 1 sec and sometimes will corrupt.

 

I confirm I didn't configure the serial port for baud rate, stop bit, parity, ... 

For reference, current (default) serial port setup :

stty -F /dev/ttyS0 -a
speed 9600 baud; rows 0; columns 0; line = 16;
intr = <undef>; quit = <undef>; erase = <undef>; kill = <undef>; eof = <undef>; eol = <undef>; eol2 = ); swtch = M-0;
start = M-s; stop = f; susp = <undef>; rprnt = <undef>; werase = <undef>; lnext = M-h; discard = <undef>; min = 0; time = 0;
-parenb -parodd -cmspar cs8 -hupcl -cstopb cread clocal -crtscts
-ignbrk -brkint ignpar -parmrk -inpck -istrip -inlcr -igncr -icrnl -ixon -ixoff -iuclc -ixany -imaxbel -iutf8
-opost -olcuc -ocrnl -onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
-isig -icanon -iexten -echo -echoe -echok -echonl -noflsh -xcase -tostop -echoprt -echoctl -echoke -flusho -extproc

 

So far, no data corruption here, as I monitor UPS data with telegraf/grafana, and any corrupted data would be clearly visible on graphs.

 

Next and hopefully last episodes :

 

1) Reboot

I've rebooted the server, and no issue after reboot, the daemon restarts properly and immediately gets good values from the UPS

 

2) Full power outage simulation

- I set "Battery level to initiate shutdown (%)" to 85% to avoid discharging the UPS battery too much

- I've unplugged the UPS to simulate a power outage.

- Unraid server orderly shutdown was initiated @ 85%

- 3 minutes later (default UPS grace period of 180 seconds), the UPS turned off as expected

- when plugging the UPS back to mains, it started properly

- the Unraid server booted properly, without parity check

- UPS values read by apcupsd are correct

- "Battery level to initiate shutdown (%)" set back to 60% in my case for normal operation

- the UPS battery is slowly recharging (94% after 2 hours)

 

So, everything is working as expected now, Modbus over serial connection with the so-called "smart" serial cable (ref AP940-0625A) was definitely the way to go rather than the USB-A to USB-A supplied cable in my particular case. Don't forget to activate the Modbus protocol on the UPS from the UPS LCD panel in configuration mode (see manual). According to the manual, Modbus should be enabled by default, this was not the case for my UPS. You must have CP.1 displayed, not CP.0, under configuration mode.

 

The only limit of this solution is you must have a DB9 serial port on your motherboard. This is generally the case on server MBs, but a serial port is rarely present on modern workstations or desktop MBs. I've read on the apcupsd support list that this solution should work with a USB to serial adapter, but you may have to try various adpaters, avoiding cheap chinese crap. And the name of the tty device in the daemon setup should be something like ttyUSB0 or ttyACM0, depending on the chip in the adapter. I haven't digged further as I have an unused native serial port on my MB. If others want to experiment, I'll pass the torch to them now !

Changed Status to Solved

Changed Priority to Annoyance

That is excellent news, I'll get the cable ordered.

Coincidently, we had a power cut today, I'll be glad to not have to perform "the ritual" after each boot!

 

Many thanks for your trials and finding the solution.

14 hours ago, sturgeo said:

That is excellent news, I'll get the cable ordered.

Coincidently, we had a power cut today, I'll be glad to not have to perform "the ritual" after each boot!

 

Many thanks for your trials and finding the solution.

You're welcome, I've been fiddling with this for so long that I'm glad to share and help others get rid of this "ritual", as you say, that I very regularly forgot when rebooting ... 😉

@limetech, @jonp

 

Maybe this thread should be moved by a forum admin in the "General support" section. I initially posted it as a prerelease bug report as I discovered the issue when I installed the UPS and my server was on 6.9.0 beta30, but it's obviously Unraid version agnostic. Moreover, it's very likely an apcupsd package and/or UPS firmware issue with USB connection, as others report the same under various distros in the apcupsd general mailing list.

 

Despite, as it is the built-in Unraid solution to communicate with UPSes, I think it would make sense to have this tested workaround visible for those who'd encounter the same problems but will very likely not find this thread buried in the prerelease bug reports section...

 

Thanks in advance for considering this move for the community.

Ok @Gnomuz, I would have given up a looong time ago. Which is exactly what I did. Big thanks for keeping at it because when I got the Unraid Ultimate Dashboard up and running that incorrect UPS reporting started bugging me again and this time I found this bugreport. Reading it was a roller-coaster ride between hope and despair :-|

 

I can report that I have exactly the same problem with APC Smart-UPS 1500 (SMC1500IC), manufactured Jul 18, 2020 and equipped with FW V. 03.5. (running Unraid 6.9.0)

I got the same nonsense numbers when trying to use ModBus as UPS type and I get Server Alert that Unraid lost communication with UPS and then directly that it's restored again.

 

Running your recipe with unplugging USB, restarting daemon and then reattaching USB got those sweet correct numbers I never seen before but they do not survive a reboot.

11 hours ago, tetrapod said:

Ok @Gnomuz, I would have given up a looong time ago. Which is exactly what I did. Big thanks for keeping at it because when I got the Unraid Ultimate Dashboard up and running that incorrect UPS reporting started bugging me again and this time I found this bugreport. Reading it was a roller-coaster ride between hope and despair 😐

 

I can report that I have exactly the same problem with APC Smart-UPS 1500 (SMC1500IC), manufactured Jul 18, 2020 and equipped with FW V. 03.5. (running Unraid 6.9.0)

I got the same nonsense numbers when trying to use ModBus as UPS type and I get Server Alert that Unraid lost communication with UPS and then directly that it's restored again.

 

Running your recipe with unplugging USB, restarting daemon and then reattaching USB got those sweet correct numbers I never seen before but they do not survive a reboot.

 

Thanks for praising my perseverance, I hate giving up, maybe a kind of OCD, but it's sometimes useful

 

The "magic recipe" for getting correct numbers was only a poor workaround for me, as t didn't survive a reboot as you mentioned. The issue we face is a apcupsd bug, reported on the apcupsd mailing list on various OSes.

 

But if you read my recent posts since late March 2021, you'll see I've now implemented a working solution which consists in replacing the USB connection with a good old RS232 serial connection between the UPS and the server ("Smart" cable ref AP940-0625A).  If you have a DB9 serial port available on your server and you're ready to invest 30€ in the cable (amazon Europe price), I would definitely recommend to go this way and forget about it forever !

Edited April 13, 2021 by Gnomuz

Yeah, I saw that. Should have put in my answer that my motherboard doesn't have RS232. Maybe I should get one of these?

at in my 

If you have a free pcie 1x slot on your motherboard, that will do the trick. The other route would be a USB to rs232 adapter, but there are  many compatibility issues as most of these so-called FTDI adapters are Chinese cheap crap. So stick with the add-on card if you can.

 

10 minutes ago, Gnomuz said:

If you have a free pcie 1x slot on your motherboard, that will do the trick. The other route would be a USB to rs232 adapter, but there are  many compatibility issues as most of these so-called FTDI adapters are Chinese cheap crap. So stick with the add-on card if you can.

 

https://www.tripplite.com/keyspan-high-speed-usb-to-serial-adapter~USA19HS

That specific model has a very good reputation.

I agree the USB-RS232 adapters with genuine FTDI chips are fine, and this product seems to be serious. The problem is it costs around 50€ here in France, so the add-on card would be cheaper for @tetrapod if he has a free Pcie slot.

Many, many thanks to the OP for starting this thread and persevering to a solution.  I recently obtained an APC SMT1500C and was experiencing the same issues.  This thread saved me countless hours of troubleshooting.  I could not find the cable indicated (AP940-0625A) at a reasonable price but I did find another cable that works, AP940-1525A, at a price I could stomach.  The cables are equivalent, the only difference being that the AP940-1525A is 15 feet versus 6 feet for the AP940-0625A.  So this provides an alternative for anyone having trouble finding the cables.  I can confirm that the AP940-1525A is working in my setup.

 

Thanks again

D

Much water have flown under the bridges and I'm finaly serverside again.

 

I tried the USB-RS232 adapter so kindly suggested by @DigitalMaestro and @Gnomuz. Unraid rightly identifies the device

root@:/mnt/user/appdata# lsusb
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 0781:5571 SanDisk Corp. Cruzer Fit
Bus 003 Device 002: ID 8087:0029 Intel Corp. 
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 06cd:0121 Keyspan USA-19hs serial adapter
Bus 001 Device 002: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

 

I'm having problems finding the right settings for the UPS deamond, BUT my bigger problem is that I'll get the original problem where the device path keeps on changing. I started at 002, but keeps changing one up:

root:/mnt/user/appdata# ls /dev/bus/usb/
001/  002/  003/  004/  005/  006/

 

I think adding a PCI serial port card is my next move. Just hope it'll arrive before I have to leave again.

I'm also waiting for the last part of a PiKWM-switch. I have high hopes for that to be a lifesaving device for my situation where I seldom are server side and have to cross four country boarders to be near my baby 😐

 

To Be Continued...

Hi, I know this topic started while ago but I found a solution to this problem adding a delay of 30 sec on the apcupsd.service.

 

Go to 

/lib/systemd/system

 

and change the file

apcupsd.service

 

by adding:

 

[Service]
RestartSec=30

 

On my Proxmox server it works perfectly.

/lib is in RAM like the rest of the OS so changes there won't survive reboot. 

 

Reapplying those changes at each boot in the go file might be too late in the boot, I don't know. 

1 minute ago, trurl said:

/lib is in RAM like the rest of the OS so changes there won't survive reboot. 

 

Reapplying those changes at each boot in the go file might be too late in the boot, I don't know. 

 

On my Proxmox server the configuration is permanent. Non need to reapply each time.

I just checked, as you can see below:

 

[Unit]
# network-online is really needed, otherwise there are problems with snmp
# -> 865620
After=network-online.target
Description=UPS power management daemon
Documentation=man:apcupsd(8)

[Service]
RestartSec=30
ExecStartPre=/lib/apcupsd/prestart
ExecStart=/sbin/apcupsd
Type=forking
KillMode=process
PIDFile=/var/run/apcupsd.pid

[Install]
WantedBy=multi-user.target

 

Just now, tamet83 said:

 

On my Proxmox server the configuration is permanent. Non need to reapply each time.

I just checked, as you can see below:

 

[Unit]
# network-online is really needed, otherwise there are problems with snmp
# -> 865620
After=network-online.target
Description=UPS power management daemon
Documentation=man:apcupsd(8)

[Service]
RestartSec=30
ExecStartPre=/lib/apcupsd/prestart
ExecStart=/sbin/apcupsd
Type=forking
KillMode=process
PIDFile=/var/run/apcupsd.pid

[Install]
WantedBy=multi-user.target

 

Unraid does not use systemd

Ok sorry, I didn't know it.

I thought it could be helpful.

 

Do you maybe know if there is a file named apcupsd.service somewhere?

Thanks for this great discussion/possible solutions! I've been hating having to unplug/plug the usb after every restart. @tamet83 did you find anything else about making the restart delay change on an unraid system? Cheers!

On 4/15/2021 at 5:19 PM, Gnomuz said:

I agree the USB-RS232 adapters with genuine FTDI chips are fine, and this product seems to be serious. The problem is it costs around 50€ here in France, so the add-on card would be cheaper for @tetrapod if he has a free Pcie slot.

Was wondering what you used for the Device for your setup. 

I've purchased a similar cable but am having a tough time locating the actual /dev/ entry to put here. 
I have seen it connecting in system logs: USB device number 14 using xhci_hcd 

But can't find any mentioned of a tty number to reference. 

Cheers for any assistance anyone else can provide as well!