MyServers Plugin causing System to Freeze

[csmccarron]

This may just be a coincidence, but after I installed the MyServers Plugin my system has locked up twice.  This is the only thing that has changed.  Prior to installing the plugin system ran for over 50 days with no issues.  The only hardware that has changed in my drives for the array.  I had to rebuild the array.  Any ideas on what the issue may be.  Diagnostic data is attached.  I have removed the plugin to see if the issue goes away.

 

It seems it locked up around Aug 19 00:20:13.

 

Aug 18 22:12:59 mccserverur01 sshd[2112]: Connection from 10.205.1.5 port 34262 on 10.205.1.6 port 22 rdomain ""
Aug 18 22:12:59 mccserverur01 sshd[2112]: error: kex_exchange_identification: Connection closed by remote host
Aug 18 22:12:59 mccserverur01 sshd[2112]: Connection closed by 10.205.1.5 port 34262
Aug 18 22:12:59 mccserverur01 in.telnetd[2110]: connect from 10.205.1.5 (10.205.1.5)
Aug 18 22:12:59 mccserverur01 vsftpd[2111]: connect from 10.205.1.5 (10.205.1.5)
Aug 18 22:12:59 mccserverur01 telnetd[2110]: ttloop: peer died: EOF
Aug 18 22:13:10 mccserverur01 smbd[2114]: [2021/08/18 22:13:10.215462,  0] ../../source3/smbd/process.c:341(read_packet_remainder)
Aug 18 22:13:10 mccserverur01 smbd[2114]:   read_fd_with_timeout failed for client 10.205.1.5 read error = NT_STATUS_END_OF_FILE.
Aug 19 00:20:01 mccserverur01 sSMTP[30668]: Creating SSL connection to host
Aug 19 00:20:01 mccserverur01 sSMTP[30668]: SSL connection using TLS_AES_256_GCM_SHA384
Aug 19 00:20:01 mccserverur01 sSMTP[30668]: Authorization failed (535 5.7.8 https://support.google.com/mail/?p=BadCredentials i16sm896282qtq.52 - gsmtp)
Aug 19 00:20:13 mccserverur01 flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update
Aug 19 19:11:34 mccserverur01 kernel: microcode: microcode updated early to revision 0x71a, date = 2020-03-24
Aug 19 19:11:34 mccserverur01 kernel: Linux version 5.10.28-Unraid (root@Develop) (gcc (GCC) 9.3.0, GNU ld version 2.33.1-slack15) #1 SMP Wed Apr 7 08:23:18 PDT 2021
Aug 19 19:11:34 mccserverur01 kernel: Command line: BOOT_IMAGE=/bzimage isolcpus=2,3,4,5,6,7,8,9,10,11,18,19,20,21,22,23,24,25,26,27 vfio-pci.ids=1b6f:7052 intel_iommu=on iommu=pt initrd=/bzroot,/bzroot-gui
 

Thank you

mccserverur01-diagnostics-20210819-2248.zip

[ljm42]

The My Servers plugin is not included in these diagnostics, so there won't be any useful log data to look at. That is fine though if you'd like run the system for a while to check the stability. Be sure to fully uninstall the plugin:

  https://wiki.unraid.net/My_Servers#Uninstalling_the_plugin 

 

The log snippet you highlighted in red is saying that the system is trying to send an email but you haven't configured it. This is part of the base OS, not My Servers. See Settings -> Notification Settings -> SMTP Settings.

[csmccarron]

Thank you for the reply.  

 

This was the last thing executed, what is this doing?

Aug 19 00:20:13 mccserverur01 flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

Edited August 20, 2021 by csmccarron

[ljm42]

I don't see those logs in your diagnostics, but this line:

  Aug 19 00:20:13 mccserverur01 flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

means that flash backup detected that a change was made to your flash drive, so it kicked off a backup.

[csmccarron]

That was the last thing that happed before it locked up.  This is when I rebooted the server later the same day.  I have the log mirrored to the USB boot drive.

 

Aug 19 19:11:34 mccserverur01 kernel: microcode: microcode updated early to revision 0x71a, date = 2020-03-24

[ljm42]

 

Not sure what that is, having a few lines before that might help. It is definitely not My Servers.

 

Is the My Servers plugin still uninstalled? Just trying to understand if it is still crashing without the plugin.

[csmccarron]

I have the My Servers plugin uninstalled currently.  Server has been running for 3 days 22 hours.  So far no more issues but it needs to run longer to verify that it is stable.  In the past, I have been able to run the server without a reboot between updates, 100+ days without issues.

 

I have attached the complete log file.

syslog

[csmccarron]

Bump

[ljm42]

I'm sorry, is there new information?

[csmccarron]

On 8/23/2021 at 5:26 PM, csmccarron said:

I have the My Servers plugin uninstalled currently.  Server has been running for 3 days 22 hours.  So far no more issues but it needs to run longer to verify that it is stable.  In the past, I have been able to run the server without a reboot between updates, 100+ days without issues.

 

I have attached the complete log file.

syslog 216.1 kB · 1 download

6 days and counting with no My Servers plugin and no freezes.

 

 

Last thing that happed prior to freezing up:

Aug 19 00:20:13 mccserverur01 flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup.php update

 

First entry after I reset the server:
Aug 19 19:11:34 mccserverur01 kernel: microcode: microcode updated early to revision 0x71a, date = 2020-03-24

 

log_prior_to_freeze.log

[ljm42]

If you would like to try again, please deactivate the flash backup since that was implicated in the last crash.  I'm not sure how it could cause an issue, but let's see how the system runs for a few days before re-enabling it.

[csmccarron]

Was wondering what would happen if internet was lost during the backup?  Would that cause the system to freeze or something?  I will enable this when I get back from my trip and post my finding here.

 

Thank you

[ljm42]

That is not something I would expect. TBH it is very difficult to think of any scenario where an application like flash backup or the unraid-api could cause the system to freeze. But let's put them back one at a time and see how it goes.

[csmccarron]

So my server has been up now for 17 days, no crash.  So what do you want me to enable first?  For now I am just going to install the plugin and leave that not signed in for about 7 days.  After that I will sign in and disable the backup.

Edited September 5, 2021 by csmccarron

[ljm42]

Sounds like a plan

[csmccarron]

Server has been up now for 26 days and 13 hours.  I have had the MyServer plugin install but not signed in now since September 5 2021 with no issues.  I am going to sign in but leave the backup disabled and see what happens.  I am also leaving Remote access disabled for now.

 

Edited September 15, 2021 by csmccarron

[csmccarron]

Server has been up now for 1 Month 20 Days 21 Hours 14 Minutes without crashing.  Just enabled Allow Remote Access and will try this for a few weeks.  The next step will be to turn the online backup on.

 

 

[spyrosj]

Any updates on this? I've been trying to determine if this has been affecting my server as well for the most part of this year.

[OmgImAlexis]

9 minutes ago, spyrosj said:

Any updates on this? I've been trying to determine if this has been affecting my server as well for the most part of this year.

Please create a separate post for your issue and include some more info such as plugin version installed, what happened and logs if you have them.

[csmccarron]

I have everything enabled except for the online backup and my server has not crashed.  I need to enable the online backup.  When I first installed the plugin the last entry in the log file was backup related and the server locked up.  Since disabling the online backup my server has been stable.

[csmccarron]

I am off this week and intown, so I am going to enable the Flash Backup and see what happens.  I am running UnRaid 6.9.2 and my My Servers plugin is up-to-date.  As of right now the Flash Backup is Active.  Due to a recent power outage my server has been up for Uptime 7 days 9 hours 38 minutes.  Hopefully it will not lockup.

 

 

[csmccarron]

Server has been stable, no lock ups or crashes.  If anything changes I will update this tread.  The only issue I have with the plugin is the "Graphql is offline" error.