[Plugin] NUT v2 - Network UPS Tools

thirtythreemangos · June 4

@Rysz I've done one test with it, where I picked settings, pulled the UPS power cord from the wall to test battery functioning, and then ultimately the graceful shutdown. It did shutdown properly 1 minute later, and the data/stats do show on the Dashboard page of unRAID, so yeah I think it still works fine. I just was unsure of what the error messages meant and if I was set up for an issue down the road or something.

Rysz · June 4

7 minutes ago, thirtythreemangos said:

@Rysz I've done one test with it, where I picked settings, pulled the UPS power cord from the wall to test battery functioning, and then ultimately the graceful shutdown. It did shutdown properly 1 minute later, and the data/stats do show on the Dashboard page of unRAID, so yeah I think it still works fine. I just was unsure of what the error messages meant and if I was set up for an issue down the road or something.

In that case you can just ignore the messages.

If they bother you a lot, you can set up "Rule-Based Syslog Filters" in NUT Settings.

You might have to tune the filter configuration file a bit to match those lines you don't want.

thirtythreemangos · June 4

@RyszSounds good. I'll probably just ignore it for now, like I mentioned I only happened to stumble across the logs, not really part of my normal workflow. Perhaps if something changes to the workflow in the future then I'll set up those filter rule things. Thank you!!!

JonathanM · June 4

4 hours ago, thirtythreemangos said:

pulled the UPS power cord from the wall to test battery functioning

That's not good. The ground connection needs to stay connected so the voltage produced by the backup stays stable. To test function you need to cut the power without unplugging it. Temporarily using a power strip or turning off the circuit breaker is much safer.

Masterwishx · June 6

@Rysz Today i had a lot of time power off/on at home and found my server with NUT not goes to power off + disable ups power after 10min as in settings:

drwatson · June 7

I am having a problem starting the NUT service for my UPS. I am not quite sure what I'm missing, but I've tried to follow other's advice and try different drivers other than usbhid-ups, but none seem to work.

Can someone help me figure out what I need to change to get this working?

I am running nut-2.8.0-x86_64-2stable.ssl11 on Unraid Version 6.11.5. I have a USB connected CyberPower UPS (cp1500 AVR LCD) only servicing the Unraid server (no master/slave required).

When I scan for the ups driver, I get the following:

[nutdev1]
driver = "usbhid-ups"
port = "auto"
vendorid = "0764"
productid = "0501"
bus = "005"

This jives with the USB Diagnostics output of:

Bus 005 Device 003: ID 0764:0501 Cyber Power System, Inc. CP1500 AVR UPS

When I try and start the service, the syslog indicates the following and NUT doesn't start:

Jun  6 22:34:37 Ceti rc.nut: Network UPS Tools - Generic HID driver 0.47 (2.8.0)
Jun  6 22:34:37 Ceti rc.nut: USB communication driver (libusb 1.0) 0.43
Jun  6 22:34:37 Ceti rc.nut: libusb1: Could not open any HID devices: insufficient permissions on everything
Jun  6 22:34:37 Ceti rc.nut: No matching HID UPS found
Jun  6 22:34:37 Ceti rc.nut: Driver failed to start (exit status=1)
Jun  6 22:34:37 Ceti rc.nut: Network UPS Tools - UPS driver controller 2.8.0
Jun  6 22:34:45 Ceti  ool www[2953]: /usr/local/emhttp/plugins/nut-dw/scripts/write_config
Jun  6 22:34:45 Ceti rc.nut: Writing NUT configuration...
Jun  6 22:34:48 Ceti rc.nut: Updating permissions for NUT...
Jun  6 22:34:48 Ceti rc.nut: Checking if the NUT Runtime Statistics Module should be enabled...
Jun  6 22:34:48 Ceti rc.nut: Disabling the NUT Runtime Statistics Module...

Here is a screen cap of the settings in the UI:

Rysz · June 7

18 hours ago, Masterwishx said:

@Rysz Today i had a lot of time power off/on at home and found my server with NUT not goes to power off + disable ups power after 10min as in settings:

I just tested a 10 minute timeout on my server, it shut down just fine... not sure what happened there.

7 hours ago, drwatson said:
I am having a problem starting the NUT service for my UPS. I am not quite sure what I'm missing, but I've tried to follow other's advice and try different drivers other than usbhid-ups, but none seem to work.

Can someone help me figure out what I need to change to get this working?

I am running nut-2.8.0-x86_64-2stable.ssl11 on Unraid Version 6.11.5. I have a USB connected CyberPower UPS (cp1500 AVR LCD) only servicing the Unraid server (no master/slave required).

When I scan for the ups driver, I get the following:
[nutdev1]
driver = "usbhid-ups"
port = "auto"
vendorid = "0764"
productid = "0501"
bus = "005"
This jives with the USB Diagnostics output of:
Bus 005 Device 003: ID 0764:0501 Cyber Power System, Inc. CP1500 AVR UPS
When I try and start the service, the syslog indicates the following and NUT doesn't start:
Jun  6 22:34:37 Ceti rc.nut: Network UPS Tools - Generic HID driver 0.47 (2.8.0)
Jun  6 22:34:37 Ceti rc.nut: USB communication driver (libusb 1.0) 0.43
Jun  6 22:34:37 Ceti rc.nut: libusb1: Could not open any HID devices: insufficient permissions on everything
Jun  6 22:34:37 Ceti rc.nut: No matching HID UPS found
Jun  6 22:34:37 Ceti rc.nut: Driver failed to start (exit status=1)
Jun  6 22:34:37 Ceti rc.nut: Network UPS Tools - UPS driver controller 2.8.0
Jun  6 22:34:45 Ceti  ool www[2953]: /usr/local/emhttp/plugins/nut-dw/scripts/write_config
Jun  6 22:34:45 Ceti rc.nut: Writing NUT configuration...
Jun  6 22:34:48 Ceti rc.nut: Updating permissions for NUT...
Jun  6 22:34:48 Ceti rc.nut: Checking if the NUT Runtime Statistics Module should be enabled...
Jun  6 22:34:48 Ceti rc.nut: Disabling the NUT Runtime Statistics Module...
Here is a screen cap of the settings in the UI:

Please post the NUT Debug Package found in NUT Settings.

thirtythreemangos · June 7

On 6/4/2024 at 4:49 PM, JonathanM said:

That's not good. The ground connection needs to stay connected so the voltage produced by the backup stays stable. To test function you need to cut the power without unplugging it. Temporarily using a power strip or turning off the circuit breaker is much safer.

'Preciate the input!

Masterwishx · June 8

21 hours ago, Rysz said:

I just tested a 10 minute timeout on my server, it shut down just fine... not sure what happened there.

I'm also tested befor it was working fine. But if power off/on many time and timer counter on/off then somehow it's broked

Rysz · June 8

8 hours ago, Masterwishx said:

I'm also tested befor it was working fine. But if power off/on many time and timer counter on/off then somehow it's broked

Shouldn't happen even with lots of power cycling going on, but hard to diagnose after the fact. In any case I couldn't reproduce this on my testing machines, so hopefully an isolated incident not to be repeated.

Masterwishx · June 9

15 hours ago, Rysz said:

Shouldn't happen even with lots of power cycling going on, but hard to diagnose after the fact. In any case I couldn't reproduce this on my testing machines, so hopefully an isolated incident not to be repeated.

just in case i will want to simulate this incident ( i can switch power off/on in computers space by switches on the power wall )

i need UPS Driver Debug Level = debug and maybe set 5 min to power off for test or something else ?

fezster · June 9

On 6/4/2024 at 10:05 AM, Rysz said:

There's certainly no harm in it, although data staleness on its own is a different problem, in your case you're having actual disconnects where the USB device disappears and that in turn causes the data staleness (as seen by the driver). I've only ever seen this happen when some kind of power saving software was in effect on the server (powertop, ...) or the USB cable/USB port were broken (USB port either on the server side or UPS side). It would be interesting, if you have another machine at your disposal, to see if your UPS also disconnects there (e.g. on a Windows machine if you'd hear the connect/disconnect sounds). But probably hard to pinpoint exactly since it appears only sporadically.

@Rysz Given your expertise on this, would like some advice. From my previous posts, you may know I have an unconventional setup - Im running Unraid on Esxi as a VM and I think this may the cause of my USB disconnects.

When I had the same data staleness this time, rather than unplug the USB, I simply rebooted the Unraid VM. And the UPS worked correctly after the reboot. This tells me it is not a problem with the cable/connection and (most likely) not a problem with a disconnection on the host machine either. Rather some kind of Virtual Machine oddity.

Here's the advice question. I came across these commands to force a USB disconnect/reconnect in UNRAID:

echo 2-2.1 > /sys/bus/usb/drivers/usb/unbind

echo 2-2.1 > /sys/bus/usb/drivers/usb/bind

#restart NUT

/etc/rc.d/rc.nut restart 2>&1 | logger

I'm going to try this the next time the problem occurs and hopefully automate it, if it works. My question is - do you think this might work (given what I said above)? And/or do you have any other advice on how to force the USB to reconnect (similar to what would happen when Unraid reboots)?

Thank you!

fezster · June 9

@Rysz Different question here to above. I had NUT restarting every hour, as I said in my post a few days ago, to see if that would resolve the staleness/disconnects. This restart has been working fine for the past few days.

However - this morning, I realised my UPS had reported on battery and the server had shutdown. I did not have any power outage. Looking at the previous syslog (which is helpfully saved on the flash), it looks like after NUT restarted, the UPS reported on battery for some reason. Have you ever come across this?

Jun  9 08:47:01 SM1 root: Stopping the NUT services...
Jun  9 08:47:01 SM1 upsmon[2921]: Signal 15: exiting
Jun  9 08:47:01 SM1 root: Network UPS Tools upsmon 2.8.2
Jun  9 08:47:01 SM1 upsd[2918]: User [email protected] logged out from UPS [ups]
Jun  9 08:47:02 SM1 upsd[2918]: mainloop: Interrupted system call
Jun  9 08:47:02 SM1 upsd[2918]: Signal 15: exiting
Jun  9 08:47:02 SM1 root: Network UPS Tools upsd 2.8.2
Jun  9 08:47:04 SM1 usbhid-ups[2912]: Signal 15: exiting
Jun  9 08:47:04 SM1 usbhid-ups[2912]: WARNING: send_to_all: write 38 bytes to socket 13 failed (ret=-1), disconnecting: Broken pipe
Jun  9 08:47:05 SM1 root: fopen /var/run/nut/usbhid-ups-ups.pid: No such file or directory
Jun  9 08:47:05 SM1 root: Network UPS Tools - UPS driver controller 2.8.2
Jun  9 08:47:08 SM1 root: WARNING: NUT was user-configured to disable power management for all USB devices.
Jun  9 08:47:08 SM1 root: WARNING: NUT is now forcing all USB devices to permanent [on] power state as requested...
Jun  9 08:47:09 SM1 root: Writing NUT configuration...
Jun  9 08:47:11 SM1 root: Updating permissions for NUT...
Jun  9 08:47:11 SM1 root: Checking if the NUT Runtime Statistics Module should be enabled...
Jun  9 08:47:11 SM1 root: Disabling the NUT Runtime Statistics Module...
Jun  9 08:47:12 SM1 root: Using subdriver: CyberPower HID 0.80
Jun  9 08:47:12 SM1 root: Network UPS Tools - Generic HID driver 0.53 (2.8.2)
Jun  9 08:47:12 SM1 root: USB communication driver (libusb 1.0) 0.47
Jun  9 08:47:14 SM1 root: using 'battery.charge' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[19996]: using 'battery.charge' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[19996]: using 'battery.runtime' to set battery low state
Jun  9 08:47:14 SM1 root: using 'battery.runtime' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[20002]: Startup successful
Jun  9 08:47:14 SM1 root: Network UPS Tools - UPS driver controller 2.8.2
Jun  9 08:47:15 SM1 root: fopen /var/run/nut/upsd.pid: No such file or directory
Jun  9 08:47:15 SM1 root: Could not find PID file '/var/run/nut/upsd.pid' to see if previous upsd instance is already running!
Jun  9 08:47:15 SM1 root: listening on 0.0.0.0 port 3493
Jun  9 08:47:15 SM1 upsd[20007]: listening on 0.0.0.0 port 3493
Jun  9 08:47:15 SM1 upsd[20007]: Connected to UPS [ups]: usbhid-ups-ups
Jun  9 08:47:15 SM1 root: Connected to UPS [ups]: usbhid-ups-ups
Jun  9 08:47:15 SM1 usbhid-ups[20002]: sock_connect: enabling asynchronous mode (auto)
Jun  9 08:47:15 SM1 upsd[20007]: Found 1 UPS defined in ups.conf
Jun  9 08:47:15 SM1 root: Found 1 UPS defined in ups.conf
Jun  9 08:47:15 SM1 upsd[20008]: Startup successful
Jun  9 08:47:15 SM1 root: fopen /var/run/nut/upsmon.pid: No such file or directory
Jun  9 08:47:15 SM1 root: Could not find PID file to see if previous upsmon instance is already running!
Jun  9 08:47:15 SM1 root: UPS: [email protected] (primary) (power value 1)
Jun  9 08:47:15 SM1 root: Using power down flag file /etc/nut/killpower
Jun  9 08:47:15 SM1 upsmon[20011]: Startup successful
Jun  9 08:47:15 SM1 upsmon[20011]: Warning: running as one big root process by request (upsmon -p)
Jun  9 08:47:15 SM1 upsd[20008]: User [email protected] logged into UPS [ups]
Jun  9 08:47:15 SM1 upsmon[20011]: UPS [email protected] on battery
Jun  9 08:47:15 SM1 nut-notify: [ups] UPS is on battery. The system will shutdown in 180 seconds.

Rysz · June 9

27 minutes ago, fezster said:
@Rysz Given your expertise on this, would like some advice. From my previous posts, you may know I have an unconventional setup - Im running Unraid on Esxi as a VM and I think this may the cause of my USB disconnects.

When I had the same data staleness this time, rather than unplug the USB, I simply rebooted the Unraid VM. And the UPS worked correctly after the reboot. This tells me it is not a problem with the cable/connection and (most likely) not a problem with a disconnection on the host machine either. Rather some kind of Virtual Machine oddity.

Here's the advice question. I came across these commands to force a USB disconnect/reconnect in UNRAID:
echo 2-2.1 > /sys/bus/usb/drivers/usb/unbind

echo 2-2.1 > /sys/bus/usb/drivers/usb/bind

#restart NUT

/etc/rc.d/rc.nut restart 2>&1 | logger
I'm going to try this the next time the problem occurs and hopefully automate it, if it works. My question is - do you think this might work (given what I said above)? And/or do you have any other advice on how to force the USB to reconnect (similar to what would happen when Unraid reboots)?

Thank you!

I can picture this working for your situation, although I've not tried it myself.

What you can also try instead of restarting NUT after is the following procedure:

Stop NUT first via /etc/rc.d/rc.nut stop 2>&1 | logger
sleep 1
Run your USB connect/disconnect commands
sleep 1
Start NUT after via /etc/rc.d/rc.nut start 2>&1 | logger

Just in case the driver doesn't like being woken up so quick after the USB connect/disconnect commands.

The only other command I can think of, to force a USB re-attachment without physically reconnecting, would be restarting the UDEV... generally not recommended and I'm not sure if it would have other negative fallout:

/etc/rc.d/rc.udev restart 2>&1 | logger

21 minutes ago, fezster said:

@Rysz Different question here to above. I had NUT restarting every hour, as I said in my post a few days ago, to see if that would resolve the staleness/disconnects. This restart has been working fine for the past few days.

However - this morning, I realised my UPS had reported on battery and the server had shutdown. I did not have any power outage. Looking at the previous syslog (which is helpfully saved on the flash), it looks like after NUT restarted, the UPS reported on battery for some reason. Have you ever come across this?

Jun  9 08:47:01 SM1 root: Stopping the NUT services...
Jun  9 08:47:01 SM1 upsmon[2921]: Signal 15: exiting
Jun  9 08:47:01 SM1 root: Network UPS Tools upsmon 2.8.2
Jun  9 08:47:01 SM1 upsd[2918]: User [email protected] logged out from UPS [ups]
Jun  9 08:47:02 SM1 upsd[2918]: mainloop: Interrupted system call
Jun  9 08:47:02 SM1 upsd[2918]: Signal 15: exiting
Jun  9 08:47:02 SM1 root: Network UPS Tools upsd 2.8.2
Jun  9 08:47:04 SM1 usbhid-ups[2912]: Signal 15: exiting
Jun  9 08:47:04 SM1 usbhid-ups[2912]: WARNING: send_to_all: write 38 bytes to socket 13 failed (ret=-1), disconnecting: Broken pipe
Jun  9 08:47:05 SM1 root: fopen /var/run/nut/usbhid-ups-ups.pid: No such file or directory
Jun  9 08:47:05 SM1 root: Network UPS Tools - UPS driver controller 2.8.2
Jun  9 08:47:08 SM1 root: WARNING: NUT was user-configured to disable power management for all USB devices.
Jun  9 08:47:08 SM1 root: WARNING: NUT is now forcing all USB devices to permanent [on] power state as requested...
Jun  9 08:47:09 SM1 root: Writing NUT configuration...
Jun  9 08:47:11 SM1 root: Updating permissions for NUT...
Jun  9 08:47:11 SM1 root: Checking if the NUT Runtime Statistics Module should be enabled...
Jun  9 08:47:11 SM1 root: Disabling the NUT Runtime Statistics Module...
Jun  9 08:47:12 SM1 root: Using subdriver: CyberPower HID 0.80
Jun  9 08:47:12 SM1 root: Network UPS Tools - Generic HID driver 0.53 (2.8.2)
Jun  9 08:47:12 SM1 root: USB communication driver (libusb 1.0) 0.47
Jun  9 08:47:14 SM1 root: using 'battery.charge' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[19996]: using 'battery.charge' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[19996]: using 'battery.runtime' to set battery low state
Jun  9 08:47:14 SM1 root: using 'battery.runtime' to set battery low state
Jun  9 08:47:14 SM1 usbhid-ups[20002]: Startup successful
Jun  9 08:47:14 SM1 root: Network UPS Tools - UPS driver controller 2.8.2
Jun  9 08:47:15 SM1 root: fopen /var/run/nut/upsd.pid: No such file or directory
Jun  9 08:47:15 SM1 root: Could not find PID file '/var/run/nut/upsd.pid' to see if previous upsd instance is already running!
Jun  9 08:47:15 SM1 root: listening on 0.0.0.0 port 3493
Jun  9 08:47:15 SM1 upsd[20007]: listening on 0.0.0.0 port 3493
Jun  9 08:47:15 SM1 upsd[20007]: Connected to UPS [ups]: usbhid-ups-ups
Jun  9 08:47:15 SM1 root: Connected to UPS [ups]: usbhid-ups-ups
Jun  9 08:47:15 SM1 usbhid-ups[20002]: sock_connect: enabling asynchronous mode (auto)
Jun  9 08:47:15 SM1 upsd[20007]: Found 1 UPS defined in ups.conf
Jun  9 08:47:15 SM1 root: Found 1 UPS defined in ups.conf
Jun  9 08:47:15 SM1 upsd[20008]: Startup successful
Jun  9 08:47:15 SM1 root: fopen /var/run/nut/upsmon.pid: No such file or directory
Jun  9 08:47:15 SM1 root: Could not find PID file to see if previous upsmon instance is already running!
Jun  9 08:47:15 SM1 root: UPS: [email protected] (primary) (power value 1)
Jun  9 08:47:15 SM1 root: Using power down flag file /etc/nut/killpower
Jun  9 08:47:15 SM1 upsmon[20011]: Startup successful
Jun  9 08:47:15 SM1 upsmon[20011]: Warning: running as one big root process by request (upsmon -p)
Jun  9 08:47:15 SM1 upsd[20008]: User [email protected] logged into UPS [ups]
Jun  9 08:47:15 SM1 upsmon[20011]: UPS [email protected] on battery
Jun  9 08:47:15 SM1 nut-notify: [ups] UPS is on battery. The system will shutdown in 180 seconds.

I'm not sure why your UPS would report that if the mains power was not gone, perhaps the input power quality (voltage/frequency) was degraded (e.g. as seen in a brownout situation) and the UPS considered it necessary to switch to battery power because of that.

The state seems to have persisted long enough (180 seconds) to cause the shutdown, but this is something that the UPS will have continuously reported to NUT (so it shouldn't have anything to do with the driver start).

Edited June 9 by Rysz

Rysz · June 9

2 hours ago, Masterwishx said:

just in case i will want to simulate this incident ( i can switch power off/on in computers space by switches on the power wall )

i need UPS Driver Debug Level = debug and maybe set 5 min to power off for test or something else ?

You don't need to change anything, the debug level is mostly for diagnosing protocol problems where NUT doesn't understand what the UPS is "saying". Ideally leave the settings as they are in production (you want to be able to rely on those after all) and just switch off the mains power and see if NUT shuts down the system after your 10 minutes.

Edited June 9 by Rysz

PartyingChair · June 15

Hello!

I was getting this error earlier:

Jun 14 21:11:02 Tower usbhid-ups[19788]: nut_libusb_get_report: Input/Output Error

Jun 14 21:11:04 Tower usbhid-ups[19788]: #012Reconnecting. If you saw "nut_libusb_get_interrupt: Input/Output Error" or similar message in the log above, try setting "pollonly" flag in "ups.conf" options section for this driver!

I added the pollonly flag, and now the error has updated too:

Jun 14 21:44:03 Tower usbhid-ups[19503]: nut_libusb_get_report: Input/Output Error

Jun 14 21:44:05 Tower usbhid-ups[19503]: nut_libusb_get_string: Pipe error

Jun 14 21:44:25 Tower usbhid-ups[19503]: nut_libusb_get_report: Input/Output Error

Is this likely a bad usb cable, or is there something else I can try before running out and spending money? If you need any diag or logs, let me know, I am happy to supply them

Thanks!

Rysz · June 16

On 6/15/2024 at 5:49 AM, PartyingChair said:

Hello!

I was getting this error earlier:

Jun 14 21:11:02 Tower usbhid-ups[19788]: nut_libusb_get_report: Input/Output Error

Jun 14 21:11:04 Tower usbhid-ups[19788]: #012Reconnecting. If you saw "nut_libusb_get_interrupt: Input/Output Error" or similar message in the log above, try setting "pollonly" flag in "ups.conf" options section for this driver!

I added the pollonly flag, and now the error has updated too:

Jun 14 21:44:03 Tower usbhid-ups[19503]: nut_libusb_get_report: Input/Output Error

Jun 14 21:44:05 Tower usbhid-ups[19503]: nut_libusb_get_string: Pipe error

Jun 14 21:44:25 Tower usbhid-ups[19503]: nut_libusb_get_report: Input/Output Error

Is this likely a bad usb cable, or is there something else I can try before running out and spending money? If you need any diag or logs, let me know, I am happy to supply them

Thanks!

If this is new and you didn't change anything, your cable is likely on the way out.

PartyingChair · June 16

4 minutes ago, Rysz said:

If this is new and you didn't change anything, your cable is likely on the way out.

Honestly, pretty sure it's been happening since I started with the UPS, I just never looked deep into it or cared since it seems to work despite the errors, but decided recently I may as well try to fix them if I can. I'll try to find a cable and try that

Masterwishx · Tuesday at 11:43 AM

On 3/28/2024 at 4:49 AM, neatgz said:

Personally, I use Eco mode, because it saves energy and reduces heat and fan noise.

How do you enabled the ECO mode ? by preesing some key on UPS or command with some config values ?

knights_of_pine · Wednesday at 04:03 AM

On 6/15/2024 at 10:05 PM, Rysz said:

If this is new and you didn't change anything, your cable is likely on the way out.

i have a brand new UPS and cable and i am getting the same errors. any other ideas?

Rysz · Wednesday at 04:48 AM

44 minutes ago, knights_of_pine said:

i have a brand new UPS and cable and i am getting the same errors. any other ideas?

Is everything else working though? There's not really much I can do from a plugin maintainer perspective, as this is a driver/communication problem. In case everything else is working, you can ignore the messages or set up the "Rule-Based Syslog Filters" in NUT Settings, which let you filter out such repetitive messages from the SYSLOG.

Edited Wednesday at 04:48 AM by Rysz

[Plugin] NUT v2 - Network UPS Tools

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

SimonF

dmacias

SimonF

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation