14 hours ago, anthem221 said:

I had a lot of log entry's that something is wrong, I fixed everything I could but the original failure still remains. 

One of the call traces is kvm related, does it still crash with VMs off?

I have two VMs running, one windows 10 I had that of to test, Still crashes. The other one....I can't really turn of because that's a pfsense...if I turn that off I can't even turn on the light in my flat 😂😂

 

But for test I can install the old router back for some days - do you think it's the pfsense? It's just running in the background with nothing attached except one pcie ethernet card.

Edited January 27, 20242 yr by anthem221

Hi all,

 

My server has been having a few unclean shutdowns. 

 

 

As you can see from the above syslog file, things stopped on the 20th and started when I powered it up this morning. I had setup Syslog server to try and capture what was going on but got nothing. Does this point to hardware failure?

 

Thanks

3 hours ago, elnevera said:

setup Syslog server to try and capture what was going on but got nothing.

Do you mean syslog server didn't work for you? Or do you mean there was nothing in the resulting syslog that seemed to explain your problem?

49 minutes ago, trurl said:

Do you mean syslog server didn't work for you? Or do you mean there was nothing in the resulting syslog that seemed to explain your problem?

Nothing notable in the Syslog, The last entry in Syslog was 15:32 on the 20th but I know that the server was still active until 03:39 on the 21st as my Scrypted container was still sending video to HomeKit until that point. 

Hello,

 

I had a few unclean shutdowns in the past couple days. I am unsure what the cause of it, but I believe it due to me pre-clearing too many drives at once?

I just had just upgraded my server to 32gb of ram and the whole array(partiy/data) been replaced with brand new drives last week. I am preclearing the old drives and getting ready to toss them since they have a number of attribute failures. Since all the changes my server seems to be unstable.

Anyone able to find out why this is happening, and double check nothing alarming in my logs?

I see these errors in the syslog.txt but unsure if it is the cause of it

diagnostics-20240401-1022.zip

Edited April 1, 20242 yr by deerow

I have had a decent number of unclean shutdowns for maybe the last 8 months.. (every power outtage even with UPS) All have taken place when power went out.. in the last instance, the power went out, and when my server was restarted it flagged as unclean and started the parity check.. I changed a few UPS settings, assuming it was this causing it.. sometime during the parity check the machine powered down and i don't know why.. Perhaps because i messed with the settings for ups shutdown... I do not believe there was an additional power outage because another computer was booted when i got home (i am gone a lot for work, long periods of time)

As far as i can tell on the flash drive under logs - their is only a diagnostics which i created myself after i booted server again.  

I had my hard drive timeout set to 60, just changed it to 420

I do not have any VM's

I did a test of the stop array command and it was maybe 30 seconds.. (but this is fresh boot of server if that matters)

I put ssh,bash settings in case it was due to a terminal session being set.. which is a possibility.

I am attaching my latest diagnostics, even though as far as i can tell it doesn't indicate shutdown success/failure.. But maybe it has important info.

I am also attaching a screenshot of my latest ups shutdown settings for comments if something should be changed.

Is there something i can do to help ensure that the syslog records the shutdown process during a power outage?

 

Hopefully i didn't miss anything in the thread when i read through it.

Thanks

 

tower-diagnostics-20240427-1416.zip

Edited April 27, 20242 yr by FrozenGamer

10 hours ago, FrozenGamer said:

As far as i can tell on the flash drive under logs - their is only a diagnostics which i created myself after i booted server again.  

If Unraid forces the shutdown there should be diags there, but you can try enabling the syslog server, and post that after the next unclean one.

Hello, 

 

after 3 months with stable server, the last 2 days started to make reboots!! and parity sync! can someone help me with the logs? im frustrated! i didnt even update the last 3 months...

 

 

syslog-192.168.4.200.log diagnostics-20240429-1032.zip

8 minutes ago, tgiannak said:

after 3 months with stable server, the last 2 days started to make reboots!!

If you mean the server is rebooting on its own, that's almost always a hardware problem.

5 minutes ago, JorgeB said:

If you mean the server is rebooting on its own, that's almost always a hardware problem.

can you point me to a direction to look based on the logs??

 

thanks Jorge

If it's a hardware issue most likely there won't be anything logged, you can start by running memtest or using for example a different PSU.

Power went out or flickered again sometime while i was gone and server was off again.. I am starting to wonder if the power supply itself isn't able to keep the load of the entire server and 32 drives without powering down.. I say this because i use a similar costco UPS on my desktop and i have been coming home to see computer powered off a few times and ups down.. (not unraid server)

 

this has never happened on unraid server as far as i know its been power outtages.  

I think desktop is lower power load, i just eliminated the ups from desktop..

If i was to choose to do a test by unplugging my unraid server ups from wall and seeing if it supports the load, i assume it would be best to do so when the server array was down and drives spinning still?  So as not to damage file integrity?  Any other ideas on how to test etc.  

I can't promise it is not a hardware issue (as in memory etc), I really dont think it is.. but it may be a ups issue?

8 hours ago, FrozenGamer said:

but it may be a ups issue?

It could be.

I have determined this is a 6 year old ups, did the simple unplug of wall to see how long it would run with just the 2 hard drive caddies and the ups shut off in 10 or so seconds, replaced it with a new ups (99 dollar CST135UC2 cyberpower from costco) and unplugged drive caddies only, says 23 minutes with that load.. 

So now that i know the ups isn't working right, i am hoping the new one will shut itself down after power down of unraid (but the 2 drive caddies cannot be shut off that i know of.)  I had read that the old similar cyberpower ups was not cable of shutting itself off afterwards.. 

 

I assume this maybe degraded the ups? or just the 6 years of use?

 

  Looking into how to recondition the batteries, but i assume i should just buy a few more from costco and not worry about it, maybe even a separate one for the caddies (32 drives) and the box amd 5900x without a graphics card)

 

Thanks everyone for all the assistance.. unraid is pretty amazing and the support in the forums is above and beyond.

 

Edit- partial answers to my own questions.. unplug worked fine and shut down the desktop with unraid.. It also triggered the auto shutdown of the UPS which set to shut down in 59 minutes.. This will not work with 32 drives connnected to it.. Looking into how to change that shutdown time with https://www.cyberpowersystems.com/product/software/power-panel-business/powerpanel-business-windows/

Edited June 4, 20242 yr by FrozenGamer
more info. notes.

Can anyone let me know if these settings look good? I am going to unplug test it now.

Hello, is anyone able to help me check my logs or tell me what should I look for. I have a couple unclean shutdowns lately and I am unsure what am i looking for in the logs on the unclean shutdowns.

I searched for "error" in the syslog.txt, but it doesnt seem to tell me much or I am unsure what I am reading at.

I have attached new logs that happened within a few hours of each other.

 

 

p.s. I had some unclean shutdowns earlier in the year too.
 

On 4/1/2024 at 10:33 AM, deerow said:

Hello,

 

I had a few unclean shutdowns in the past couple days. I am unsure what the cause of it, but I believe it due to me pre-clearing too many drives at once?

I just had just upgraded my server to 32gb of ram and the whole array(partiy/data) been replaced with brand new drives last week. I am preclearing the old drives and getting ready to toss them since they have a number of attribute failures. Since all the changes my server seems to be unstable.

Anyone able to find out why this is happening, and double check nothing alarming in my logs?

I see these errors in the syslog.txt but unsure if it is the cause of it

diagnostics-20240401-1022.zipUnavailable

 

batcomputer-diagnostics-20240714-2325.zip batcomputer-diagnostics-20240714-1918.zip

1 hour ago, deerow said:

I have a couple unclean shutdowns lately and I am unsure what am i looking for in the logs on the unclean shutdowns.

Do you mean the server is crashing and you need to force a reboot or you are doing a clean shutdown/reboot and it doesn't do it?

Need a little bit of help.  Never had a problem with this server and tonight i went to use it and noticed that it shutdown (on a big UPS) and there was no power glitch so it wasn't related to that.  I am attaching the diagnostics so hopefully someone can tell me what might have occured.  I have 2 Parity drives 8 data drives.

node804-diagnostics-20241023-2206.zip

The syslog in the diagnostics is the RAM copy and only shows what happened since the reboot.   It could be worth enabling the syslog server to get a log that survives a reboot so we can see what happened prior to the reboot in case it happens again.  
 

The mirror to flash option is the easiest to set up, but if you are worried about excessive wear on the flash drive you can put your server’s address into the Remote Server field and log to a pool or the array instead.

I would be extremely worried about the flash, this is one of the weakest links with unraid.   But I will look into remote syslog.   So far It's still rebuilding the parity without any issue.   Other than the syslog file, is there anything else to look at in the diagnostic file?

Hi, one of my 2 Unraid servers had an unclean shutdown yesterday, which I only found out hours later. I was not using Unraid actively at the time of the shutdown, but there were ~ 15 dockers running.  When I tried to use one of the dockers, it would not respond. So I looked at Unraid and it was down. The server was running, I could hear the fans. I could also ping the IP just fine. 

 

So I hit the power button for a split second, but nothing came up. In the end, I turned the server off and pulled the USB stick.  The parity.check.tuning.log tells me, that there was an unclean shutdown. The last diagnostics file is ~ 6 weeks old, so that will not be any good. 

 

The server was built by me ~ 1 year ago with brand-new parts and never moved from his spot since, temperatures are low. I am not very versed in debugging situations like this, so any hints on how to move on from here to get the server back up are very much appreciated.

 

PS: I got it back up by putting the stick in a different USB port as before. Right now, it is parity checking (for the next 8 hours). Is it normal, that I can't use my dockers during that period or did they not survive the unclean shutdown? 

Edited October 26, 20241 yr by linuxdevil

4 hours ago, linuxdevil said:

Is it normal, that I can't use my dockers during that period or did they not survive the unclean shutdown? 

All functionality should still work during a parity check although performance will be degraded.  
 

You are likely to get better informed feedback if you attach your system’s diagnostics zip file to your next post in this thread. It is always a good idea when asking questions to supply your diagnostics so we can see details of your system, how you have things configured, and the current syslog.

Can I get the file without connecting the stick to my PC? If not, I will shut the server down once the parity check is done and post the file here. I got most things up and running. My main concern now is: Why did this happen and will it likely happen again?