My server has been fine for the last year running 24/7 until today. Running the latest release 6.5.0. All server VMs went offline, when I load the web gui the plugins, dockers and VMs tabs won’t load. I restarted the server and started getting spammed with emails as seen in the attachment. 

Post your diagnostics

I can’t currently access them from the webgui. Server has completely crashed just trying to power it down cleanly if possible then I will post them after reboot if possible

Had to force powerdown the server and I have now rebooted it - everything appears alright but I would like to get to the root of what actually happened and if it is still an issue

tower-diagnostics-20180321-2303.zip

8 minutes ago, guyturner797 said:

I can’t currently access them from the webgui. Server has completely crashed just trying to power it down cleanly if possible then I will post them after reboot if possible

That sucks.  Next time if it happens and you can't grab them from the webUi, if at all possible try to get them from the terminal

diagnostics

 

Side Note:  Advanced Buttons is incompatible with 6.4.1+ (fix common problems would have told you about this), and WILL basically trash your docker apps if you ever update them.  Remove it.

I was connected to it by local terminal, ran shutdown and it completely froze with no response at all

 

Great, should have checked that my fault - anything else in the diagnostics or was it a 'blip' as it were?

 

Thank you so much

side note: I cant uninstall plugins! or update them. It says successful but reloads and doesnt change

3 minutes ago, guyturner797 said:

side note: I cant uninstall plugins! or update them.

Since the UI is working now, try to uninstall, then post diagnostics

I found downgrading, removing buttons, and then upgrading seemed to fix it. Just in-case I have attached my diagnostics.

tower-diagnostics-20180321-2338.zip

After a couple days the issue has resurfaced! I got an email saying parity check had finished then within a couple minutes the server started spewing out emails again and now the docker tab has disappeared (dockers are still running). I believe I am having some sort of issue with 6.5.0 as when I downgrade to 6.4.1 everything is fine? 

tower-diagnostics-20180323-0757.zip

Edited March 23, 20188 yr by guyturner797

Looks like you rebooted your system. Next time when it happens, try to get the diagnostics before you reboot the system. All information is stored in RAM and is lost after a reboot.

 

Ah crap sorry - when the system does this it completely freezes up and the gui is unresponsive so I can’t get a diagnostics

9 hours ago, guyturner797 said:

spewing out emails again

k.  What is the emails then.  The pic in the OP is cut-off and not much help

 

3 minutes ago, guyturner797 said:

I can’t get a diagnostics

From either SSH (Putty), or at the locally attached keyboard / monitor enter in

diagnostics

Or you can also install the Fix Common Problems plugin, then toss it into Troubleshooting Mode and do what it says after a crash.

So after downgrading to 6.4.1 from 6.5.0 the server VMs no longer crashes and the gui stays accessible but I still receive the error emails (sorry for not attaching it in full the first time I have now below.) Also some sections of the gui remain non-functional such as the plugins page and diagnostics. 

 

Under the shares tab it shows a couple errors too - I have attached a screenshot.

 

I tried running diagnostics from Putty - the console outputted the same errors as in the email and I have attached the zip file it created. Every time I run a command it outputs this:

-bash: fork: retry: Resource temporarily unavailable

 

tower-diagnostics-20180324-1827.zip

Unable to fork implies to me (but others can and may be more knowledgeable) that a limit has been hit on the number of processes currently running.  What happens if you run in safe mode?

I found that too looking on stack, I ran the commands suggested to check the number of processes and threads being used and was at 15% of the maximum 

Hey was this ever resolved? How did you get it fixed?

 

I'm having the exact same issue on 6.5.3. I was considering downgrading? Just posted to see if any feedback here.

 

On 3/25/2018 at 6:19 PM, guyturner797 said:

I found that too looking on stack, I ran the commands suggested to check the number of processes and threads being used and was at 15% of the maximum 

 

As suggested by @Squid run your system in safe mode and with docker service disabled to see if a basic set up stays stable.

 

On 6/27/2018 at 5:20 AM, JonahJ said:

Hey was this ever resolved? How did you get it fixed?

 

I'm having the exact same issue on 6.5.3. I was considering downgrading? Just posted to see if any feedback here.

 

 

I ended up making a full backup of all of my config files and then flesh installing Unraid...