[Solved] Did my ups fry usb ports during heat wave ?


potjoe

Recommended Posts

Hi all, 

 

A few weeks ago, we had a heat wave here in France, and both my server and its ups (APC Back-ups Pro 550G) runned hotter than usual (+6 C° delta). 

On August 10th, I noticed that Apcupsd reported a 96% battery charge constantly. I thought of a bug, and restarted the server. Battery ranked up to 100%. 

 

Later, Unraid reported (without email notification) a power failure at 23:11 for 2 seconds, a second one at 00:03 for a second, and a last (lethal) one at 00:08, immediately followed by a "Communications with UPS lost." report. Looking in logs, my domotic antenna got also disconnected (FTDI USB Serial Device), it was connected on the same usb bus.

The next day, neither of the two devices were recognized using their original usb ports, I had to switch to another one to get them running again. Connecting a usb stick in the assumed dead usb ports confirmed it : nothing appeared in the log as they were plugged. 

 

In the same time, from August 10th to a few days later during the heat wave, I noticed electrical flickering in my house (Tv and Light), and the ups was still reporting 96% battery charge with no outage to report. 

 

In order to prevent this in the future, I'd like to understand what happened. Thus, I have 3 questions :

 

     - Why the ups battery charge was only 96% while plugged, and can it be related to heat ? 

     - Did my ups kill a usb bus, should I be concerned regarding its lifetime or its battery, and how to avoid this situation in the future ? 

     - Could electrical flickering have caused ups to go crazy, or could the ups going mad (due to heat?) have caused elctrical flickering in the house ? 

 

I'm really concerned about my house and hardware safety at this time. Thank you for reading me.

monalisa-diagnostics-20200811-0119.zip

Edited by potjoe
Link to comment
58 minutes ago, potjoe said:

In the same time, from August 10th to a few days later during the heat wave, I noticed electrical flickering in my house (Tv and Light), and the ups was still reporting 96% battery charge with no outage to report. 

This is usually an indication that the mains voltage is fluctuating widely.  (Europe tends to have fewer of these events than the rest of the world because the configuration of your power grid!)  You may actually have voltage spikes/transients  without there being an actual power outage.  They are never good for electronic devices.   Many UPS's have circuity to suppress these voltage transients BUT some of the components used actually 'wear out' with repeated hits.  Some UPS's will go onto battery operation to hold the voltage output with specified limits.  Others only go on Battery when the mains power  is interrupted. 

 

One thing you may not realize is that most (if not all) consumer grade UPS's deliver mains power until conditions dictate that a switch to battery power is required.  It is possible that a voltage surge from the UPS did kill the USB bus.  It is also possible that a voltage surge coming out the server did the same thing. 

 

You should also realize that you probably have several 'spare' USB headers on the Motherboard.  You can purchase a bracket which has USB outlets on with a plug to connect to the headers if you need more USB ports.

 

     https://www.amazon.com/StarTech-com-Port-Female-Plate-Adapter/dp/B00015Y0FK/ref=pd_bxgy_2/141-9525448-8872023?_encoding=UTF8&pd_rd_i=B00015Y0FK&pd_rd_r=b9ce6ee4-4840-4bc5-a22a-88423a284f3c&pd_rd_w=bwQC8&pd_rd_wg=CkvvS&pf_rd_p=ce6c479b-ef53-49a6-845b-bbbf35c28dd3&pf_rd_r=VSTR93TQYK9ZQ29GR212&psc=1&refRID=VSTR93TQYK9ZQ29GR212

 

One more thing, you may not be seeing E-mail about UPS events if you do not have your Local Network devices on a UPS.  Unraid makes a single attempt to send an E-mail. If that fails because the network is down, it is only noted in the syslog. 

Edited by Frank1940
Link to comment

Thank you for your complete reply! I realize now that electric fluctuation may be the cause of my troubles. You did not mention the heat variable, do you exclude the possiblity that the ups overheated ? 

17 minutes ago, Frank1940 said:

Some UPS's will go onto battery operation to hold the voltage output with specified limits.  Others only go on Battery when the mains power  is interrupted. 

I was surprised by this behavior, since this model has an AVR feature, supposed to handle electric fluctuation without switching on battery. 

I already have an extension for USB plugged into the motherboard, Unraid flashdrive is plugged on it! Maybe I should switch for a 4 USB extension considering the low cost. 

 

Regarding Email, I received the "Communication lost with ups" alert at 00:08, but no alerts for the three precedent power failure. When testing the UPS, I usually receive the notification... 

 

Finally, do you advise me to replace the UPS unit/UPS battery/power supply after that ? 

Link to comment
52 minutes ago, potjoe said:

You did not mention the heat variable, do you exclude the possiblity that the ups overheated ? 

Heat is the enemy  of all electronics.  Normally, most electronic gear will operate satisfactory in environments that Human beings can not even tolerate.  But I doubt if your heat wave even approached what are normal highs in many parts of the world.  Phoenix, AZ,  is predicted to have a high of 106F  (41C) and that is almost typical for that city during the summer.  (Air conditioning is virtually universal in that city!) Your UPS is designed to operate at temperature found in those regions.  Now could your particular UPS have failed at the temperature it reached?  Yes.  But I am surprised that it could have fixed itself...

 

1 hour ago, potjoe said:

I was surprised by this behavior, since this model has an AVR feature, supposed to handle electric fluctuation without switching on battery. 

I googled this and found the following article. 

 

https://www.quora.com/How-does-AVR-Automatic-Voltage-Regulation-work-on-a-UPS

 

It is a somewhat brute force method for handling the type of voltage fluctuations that the third world often sees where the generation capacity is so overloaded that they can not hold the specific voltage for long periods of time.  (At one time, there was a gentleman who lived out in country in the Philippines where the power went out three to five times a DAY!!!)   I don't what exactly the power condition were/are-- A small drop in voltage due to increased loads or 2000 volt spikes caused by caused by sudden load transitions.

 

Years ago, many large data centers run on inverters 100% of time.  Mains power were used charged batteries.   Many had backup Diesel generators that were automatically started after a mains failure. 

 

By the way, consumer UPS are the bottom of the pile when it comes to protection.  You can purchase much better units intended for commercial usage but they are very, very expensive.  (If you decide to purchase  one of these, be careful as I seem to recall that their signalling protocol may not be compatible with Unraid's monitoring software.)

1 hour ago, potjoe said:

Regarding Email, I received the "Communication lost with ups" alert at 00:08, but no alerts for the three precedent power failure. When testing the UPS, I usually receive the notification... 

As I said, do you have your LAN devices (switches, router, modem) on a UPS?  If you don't, when power fails, your network will go down and your server can't send out an E-mail about the UPS warnings.  You can only test a UPS for proper behavior in this situation by turning off the power to your house and seeing what  happens. 

Link to comment

APC Back-UPS (Pro or not) line isn't exactly their higher end units.

First, they're modified sine wave, unless explicitly saying otherwise, so their power isn't as nice.   If yours is this unit... https://www.apc.com/shop/uk/en/products/APC-Power-Saving-Back-UPS-Pro-550/P-BR550GI you should note that "Waveform type: Stepped approximation to a sinewave" is listed.

Second, no UPS is perfect, sadly.  I've had massive Symmetra SY16K frames go poof if the weather is just right.

Link to comment

Thank you both of you for your detailed explanations. Now I better understand what happened and the succession of events, not specifically related to heat as I understood initially, which was mysterious to me. I guess it's just the way UPS and power fluctuations works, and I'll have to move to a higher end unit to be able to protect more efficiently my server. 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.