So I just upgraded to 6.1.9 and I'm having an issue where when I run a parity check from the "Main" tab in the UI, after about 10-15 minutes the server seems to reboot itself, although it's not actually rebooting.

 

What seems to be happening is that I am getting an "unclean shutdown" message and the array goes offline, but only when I run a parity check (as of now). I am still testing to see if it's tied to another issue entirely.

 

There was no unclean shutdown at any time. The only reason I decided to run a parity check in the first place was to make sure everything was ok, because I haven't run one in such a long time.

 

Any ideas on what could be causing this? I can post a log if you all can point me in the direction of the one to post. It seems the "system log" is cleared after the system "thinks" it's shut down, so I'm not exactly sure how to find the log prior to that occurrence.

the server seems to reboot itself, although it's not actually rebooting.

So it is not rebooting? Then you shouldn't call it rebooting.

 

I am getting an "unclean shutdown" message ...

There was no unclean shutdown at any time.

Were you next to the server when it, what you call, "rebooted"?

"Unclean shutdown" is detected, if the "super.dat" (iirc) is not (marked as) closed before shutdown.

As you can imagine, this usually happens if the server is hard reset or if the thumbdrive is not accessible to save the status of the "super.dat".

 

Any ideas on what could be causing this? I can post a log if you all can point me in the direction of the one to post. It seems the "system log" is cleared after the system "thinks" it's shut down, so I'm not exactly sure how to find the log prior to that occurrence.

If the system powers down regularly, it will save the log of the last sessions on the thumbdrive.

If it crashes, freezes or resets there will be no log.

If it crashes in a way that you still have access to the GUI, you can save a diagnostics file through the GUI.

If it crashes in a way that you still have access to the console, you can call the diagnostics routine from the console also.

 

Edit: follow the troubleshooting link in my sig to see how to grab a syslog respectively diagnostics file

Edit2: just noticed, the wiki isn't covering the diagnostic file from console - simply call "diagnostics" and check the log directory

it's not actually rebooting.

How did you determine that? Have you watched the local console to be sure? (monitor attached directly to the server, or IPMI)

 

It sounds like you have a fairly nasty crash happening, I'd run memtest from the console boot menu for 24 hours to rule out a bad stick of RAM. Switching from version 5 to 6 means going from 32 bit to 64 bit, so the way memory is accessed and used changes drastically.

 

Also, check for motherboard BIOS updates, sometimes newer software needs fixes in the BIOS.

 

The Fix Common Problems plugin can be configured to log diagnostics continuously, so if you get a clean memtest and the BIOS is the latest possible, I'd install FCP and configure it to save the log periodically so you can possibly catch it before it fully crashes.

Thanks for the replies. The reason I'm sure it didn't reboot is that I was looking at my tower, and it never appeared to reboot, and what was happening is one second it would be running a parity check, then less than 20 seconds later the UI would display 1 minute uptime and the array was stopped. There was no appearance of an actual reboot from what I could see. This happened 4 times last night.

 

I decided to turn off the UPS setting that I'd set, as that was the only thing different from my setup in 5.0.5. Once I turned that off, the parity check completed overnight with no issues. So, hopefully all is well.

 

... The only reason I decided to run a parity check in the first place was to make sure everything was ok, because I haven't run one in such a long time.

 

You did the upgrade without first doing a parity check ??!!  Definitely NOT a good idea.  You should always confirm that everything's good BEFORE making any significant configuration changes.

 

... I decided to turn off the UPS setting that I'd set, as that was the only thing different from my setup in 5.0.5. Once I turned that off, the parity check completed overnight with no issues. So, hopefully all is well.

 

Very strange.    If you're using the latest version of PowerDown, I did notice that there was an update on the 5th, followed by another update yesterday (the 6th) => so perhaps there was a bug in the update that was the cause of your issue.    Assuming you're using it, go to the Plugins tab and do a "Check for Updates" to be sure you have the latest version.

 

If you're not even using the powerdown plugin, I don't think you will get any syslog saved to flash even when you have a controlled shutdown.

 

In the absence of any other evidence, I would say the one minute uptime you mentioned is the best evidence we have that it rebooted.

 

And an unclean shutdown would seem to indicate a loss of power.

 

Perhaps some combination of your apcupsd settings and your UPS battery state caused an unclean shutdown when you had an unnoticed power glitch, and that was followed by a reboot due to your BIOS power-on settings.

 

Perhaps some combination of your apcupsd settings and your UPS battery state caused an unclean shutdown when you had an unnoticed power glitch, and that was followed by a reboot due to your BIOS power-on settings.

Just went through hell diagnosing a similar issue with my mother's main computer.  Turned out to be the power cord wasn't seated 100% correctly.

Agree it could have been a power issue either due to a loose cord OR a failed UPS battery.  How old is the battery in your UPS?    An easy way to test whether the UPS battery is to (a) shut down your server;  (b) plug in an alternative load for the UPS (e.g. a 100 or 200 watt light); and © unplug the UPS.    If the light doesn't stay on for a reasonable amount of time (e.g. at least 10 minutes) you should replace the batteries in the UPS.

 

... The only reason I decided to run a parity check in the first place was to make sure everything was ok, because I haven't run one in such a long time.

 

You did the upgrade without first doing a parity check ??!!  Definitely NOT a good idea.  You should always confirm that everything's good BEFORE making any significant configuration changes.

 

Noted, I'll be sure to do that in the future. There are so many things people say you should do that are assumed. I wish there was some definitive place where all this information was placed in a concise format. I used the 5.0 to 6.0 upgrade wiki and nowhere in there did it say I needed to do a parity check before upgrading.

... The only reason I decided to run a parity check in the first place was to make sure everything was ok, because I haven't run one in such a long time.

 

You did the upgrade without first doing a parity check ??!!  Definitely NOT a good idea.  You should always confirm that everything's good BEFORE making any significant configuration changes.

 

Noted, I'll be sure to do that in the future. There are so many things people say you should do that are assumed. I wish there was some definitive place where all this information was placed in a concise format. I used the 5.0 to 6.0 upgrade wiki and nowhere in there did it say I needed to do a parity check before upgrading.

And regularly scheduled parity checks is also recommended so you never get to a point where you "haven't run one in such a long time". Monthly is the usual setting.

 

And make sure you set up Notifications! I just posted on a thread where someone running V6 "discovered" they had a disabled disk. Notifications will tell you when something is wrong so you don't have to discover it.

I just wanted to update.

 

After the array has been online for 10 days with no issues, with the APC daemon on, I decided to replace one of my old 1TB drives with a new 3TB drive.

 

I did pre clear on another PC, and added the drive to the array, started the array to start data recovery, and then the array goes offline again.

 

It seems that there is an issue with my setup and the APC daemon being enabled during parity checks and rebuilds. Anyone every heard of this?

 

Upon seeing this "reboot" which again is not actually a reboot, just a strange unmounting of the drives suddenly in which the GUI shows 1 minutes uptime after I start the array again, I turned off the APC daemon, and the rebuild is going well now.

 

Hopefully this issue will be fixed for me when I upgrade to 6.2. Now I just have to hope there are no power failures in the next 7 hours

I've seen faulty APC UPSes that claim they are on battery and empty while working fine on main power. Try unplugging your UPS data cable and see how it works.

Ok, something crazy is going on. Immediately when I mount the array to start the rebuild, the server actually reboots itself now. I have it hooked up to a monitor and as soon as the array starts, the server reboots.

 

Any ideas? This is freaking me out.

This is my syslog.

 

http://pastebin.com/hyX9ETbz

What is the exact model of your power supply?

You can also try starting in maintenance mode, if it doesn't reboot cancel rebuild and check filesystems, a corrupt filesystem can make unRAID crash on mount.

You can also try starting in maintenance mode, if it doesn't reboot cancel rebuild and check filesystems, a corrupt filesystem can make unRAID crash on mount.

 

I tried this. It automatically rebooted once the array was started. Literally wishing seconds of the array starting.

 

So I powered it down, checked all connections. All connections seemed to be fine. I made sure the power cord was plugged in tight, even moved it to a different room with a different power cord and had the same issue.

 

I powered it down again, rebooted and now it  seems to be ok. It's been rebuilding the drive for about 2 hours now and hasn't restarted. I'm hoping it makes it through the night, and the drive rebuilds ok. I will upgrade to 6.2 tomorrow if all goes well tonight and see if that fixes my issues.

 

This is all so very strange.

 

As for my exact PSU, I will have to reply about that tomorrow. I'm afraid to get close to the tower right now fearing it might somehow crash my rebuilding process. Hell earlier the rebuild crashed when I was just browsing the WEB UI. So I'm not even leaving that open, and just checking it quickly every hour or so to see how it's coming along. I truly do not know what the cause of the problem is.

I tried this. It automatically rebooted once the array was started. Literally wishing seconds of the array starting.

 

Then it's almost certainly a hardware issue, like trurl suggested the power supply is one of main suspects.

Well the data was rebuilt with zero errors, thankfully. I'm going to back up my flash disk and then upgrade to 6.2.

 

My PSU is this, http://www.newegg.com/Product/Product.aspx?Item=N82E16817207013, purchased in June 2014.

I just upgraded to 6.2.

 

Started a parity check, and within 15 seconds the server rebooted.

 

I'd agree this sounds like a hardware issue, but why does it only creep up when I'm rebuilding a drive or checking parity? I guess the PSU should be the first check?

I'd agree this sounds like a hardware issue, but why does it only creep up when I'm rebuilding a drive or checking parity? I guess the PSU should be the first check?

Well that is the point at which are most likely applying a peak load to your power supply if that is the issue.

Agree it sounds like the PSU => when you start a parity check, the first step is to spinup ALL drives ... which puts the maximum load on your power supply -- not only is there a surge for the spinup current, but as they spin up every drive is actively reading.

 

Your PSU is a fairly inexpensive unit which may have simply experience enough capacitor degradation over the time it's been inservice that it's no longer reliable at those loads.  Certainly not the ONLY possibility; but it's the first thing I'd check ... by substituting a new, higher quality PSU.

I've replaced the PSU with a 500 watt that I had in my FreeNAS backup server.

 

So far, the parity check is running with no hiccup. It's a royal PITA to practically rebuild the computer and destroy all of my case organization, but it had to be done.

 

I'm also streaming a couple 1080p movies while doing this because before, at least on several occasions when the server didn't automatically reboot upon starting a parity check, I attempted to watch a movie from the server and it rebooted at that time, under an increased load, i suppose.

 

Hopefully this will solve all my problems. And here I was hoping this would be the last weekend I would have to spend on server issues.