I am having a problem where the system seems to lockup and it becomes unresponsive.  I can't access the web gui or telnet into it and I can't "wake up" the sytem.  By that I mean it usually goes into a sleep mode (at least the monitor goes black) after a period of inactivity and moving the mouse or using the keyboards will bring it back, but when it's locked up the mouse and/or keynoard won't get a display on the monitor.  I can't sem to find any way to get into the system.  It's done this twice now.  The first time was the night before last while it was transferring a bunch of files to it overnight.  It did it again last night, but I'm not sure if it did it while it was idle or during a parity check.  I had started a parity check before I went to bed, but I don't know when it crashed.

 

Is there some way to get a log file?  What could be the problem?  I'm still using the "special" version you made for my mobo.

25cents give you:

-check bios powersave features and make sure they are disabled.

-maybe your cpu is overheating and shuts down during load to prevent damange.

/Rene

I'll double check the power saving options in my BIOS, but I'm pretty sure they are all disabled.  As for the CPU overheating, in my experience it would either completely shutdown the system or cause a reboot not just lock up the system and it still be powered on.  Also, this complete system ran both XP and Windows Home Server without any problems.  That's not to say that unRAID might not be more demanding, becasue it might be, but nothing here would indicate to me that I have a CPU overheating problem.

 

This has now happened a third time while it was just sitting idle (at least I think it was).  How can I find the problem Tom?  Is there a way to cause unRAID to create a log file on every boot up keeping the last and current one on the flash so I can reboot and see what's causing the problem?

Probably the easiest way to do this is open a telnet window and type:

 

 tail  -f  /var/log/syslog

 

This will output the last few lines of the system log and then each time a new message is generated it will print in the telnet window.  Then if some kind of error is leading up to the crash you should be able to select/copy/paste & post it here.

I checked the BIOS and all power management options were disabled.

 

I just rebooted the unRAID server and started a telnet session with that command on my desktop.  We'll see what that yields.  I have noticed that each time the system freezes and I reboot it unRAID automatically performs a parity check.  I assumed this is a totally normal thing after a problem, but I thought I'd mention it in case it may indicate something else.

I have attacched the results of the tail command.  I copied the info from the telnet screen just now while the system is currently locked up.  Is it any help at all?

Last entry in the log is a spin-down command to device "/dev/sda".  You can look at the Devices page to see which disk this is.

 

Any idea how long after this last message your server locked up?

 

If it's related to disk spin-down, you can try disabling that and see if a lock up still occurs.  Another idea is to force drives to spin down and see if that results in a lockup:

 

hdparm  -y  /dev/sda

 

(Go to Devices page and look at identifier inside paranthesis; substitute it for 'sda' in command above for each disk.)

 

If it's not related to spin-down, could be difficult to diagnose.  I'd suspect power first, then system ram.

I'm trying to issue the spin down command through telnet.  Once I run the command should I then see the flashing indicator in the web gui?  Because it's not flashing to indicate the drive is spun down.

 

Edit. My lone SATA drive is sda.

I also ran that command on both my IDE drives and neither of them showed flashing in the web gui either.

I don't think you're looking for the lights in the gui to flash. What you are doing is disabling the spin down feature so that the drives don't spin down. Then, look to see if your machine still crashes. If it doesn't, it may have been a problem with spinning down the drive.

I don't think you're looking for the lights in the gui to flash. What you are doing is disabling the spin down feature so that the drives don't spin down. Then, look to see if your machine still crashes. If it doesn't, it may have been a problem with spinning down the drive.

 

 

Tom wrote, "Another idea is to force drives to spin down and see if that results in a lockup: hdparm  -y  /dev/sda".  So that command forces drives to spin down which is what I was trying to do, but it didn't seem to work.

 

 

Tom,

 

After running your hdparm command above and failing to see the flashing indicators in the web gui, I decided to try something.  I openned a telnet seesion and ran your tail command to display the last few entries of the log, but it seemd impossible to keep that running and still issue new commands.  So I tried to open another telnet session to the server to issue the hdparm -y command.  I figured that this would allow me to see any response to the command in the previous telnet window.  I was able to see the log respond to any action I performed in the second session (logging in and such), but when I issued the hdparm -y command on any of my hard drives, nothing showed in the log session.  There was absolutely no response to the command in that window.  In the window I actually typed the command I would get the response "Issuing standby command" or something like that once I pressed enter, but nothing to indicate a response to that command.  That is why tried to check the web gui and then a second telnet session with the log display running. 

 

Anyhow, after waiting an hour for the drives to spindown at the behest of unRAID, they all showed flashing indicators in the web gui and the system wasn't locked up.  So it doesn't appear to happen right when they spin down, but maybe once they/one have been spun down for a while?  I am going to get new power supply today or tomorrow and try that since the one in the computer is a chincey looking 250W.

Yes, sorry, what you're seeing  so far is normal.  The 'tail -f /var/log/syslog' command will display system logging messages as they occur, but most of the time when a command completes normally, nothing gets logged (makes sense right).

 

The 'hdparm -y' command directly sends the 'spindown' command to the drive, bypassing the unRAID System Management Utility, which doesn't know the drive is spun down and so you don't see the 'indicator' start to blink.

 

This was a "stab in the dark" anyway (thinking it had to do with disk spin down).  If you are using a "chincey looking 250W" power supply, that is a very likely suspect.

 

I should also add: it's perfectly ok to have multiple telnet windows open to the server simultaneously.

OK.  Thanks.  I guess that all makes sense.  I just picked up a new power supply a few minutes ago, so hopefully that fixes the problem.

Heck, I guess I misread the post. Sorry about that...just trying to help. 

Last night I replaced the power supply with an Antec 430W and so far so good.  It's been ~24 hours since I got it back up and running and that's the first time it's gone that long without crashing.  Hopefully that means it was the power supply that was the problem.  I'll report back in a couple days if it indeed solved the problem.

Well, it's back to locking up again.  It has locked up 6 times so far this weekend including once last night while I was watching a movie that was on it.  It seems to be pretty random.  It's happened during a parity sync, while everything was idle and all drives were spun down, while copy data to the system and while copying data from the system.  As I said, I purchased a new power supply and I also replaced the RAM with some from my main machine and neither has helped.  If the problem is hardware related then unRAID is extremely finicky because all this hardware ran Windows Home Server fine for weeks without a reboot.

 

This is a bummer because I love the concept of unRAID and it's the only one that really fits my needs.  I'm not sure what to do now as I don't want to go out and buy new hardware that might work.

Any other thoughts?

In re-reading the whole thread, spin down of the SATA drive still seems to be a prime suspect.  You've tried forcing a spin down, but I do not see where you have tried Tom's other suggestion, to disable spin down (forgive me if you have).  As to WinXP and Windows Home Server, they may very well not support spinning down your drives, at least in the same way, so that may be one behavioral difference from unRAID.

 

I can think of 2 more tests, that at least will eliminate this suspect.  The first is of course setting spin down to Never, on the Settings page of the web gui, and checking for successful operation for a day.

 

The second is to determine if the crashes occur one hour (or whatever your spin down delay is) after the last access to your SATA drive.  Although the crashes have 'seemed' random, does it seem possible, in thinking back over what you have been doing within an hour of each crash, that the crashes occur exactly one hour after the last read or write to the SATA drive?  I suggest re-starting the array, so that the spin down timer starts fresh for all drives, then do nothing for awhile EXCEPT a read from the SATA drive every half-hour.  Then confirm that spin down occurs correctly for all other drives, and everything still works fine.  Continue with the half-hour reads of the SATA drive for a few more hours, or at least until you feel certain it should have crashed by now.  Then stop reading, and check what happens when the SATA drive tries to spin down.  You will have confirmed or eliminated this suspect.

 

If it is a SATA spin down problem, I would check for an updated BIOS.  As mentioned elsewhere, SATA spin down seems to be fairly new in this industry, so a newer BIOS revision for your SATA chipset might very well fix the problem.  Otherwise, you can live without spin down, or perhaps Tom will have a way in the future to control spin down on an individual drive basis.  Or he or someone may have a way (through hdparm?) to manually disable spin down of only your SATA drive.

 

Hopefully, somewhere in here, there was a helpful idea.

 

Rockdawg, can you post more information about your system please?

A breakdown of the components would be a good place to start.

Yesterday I came home and it was locked up again, so I went into the BIOS and set it to "Safe Defaults" just in case.  This also turned off SMART reporting on the IDE channels so now my IDE drive isn't showing a temp in the web interface.  The only BIOS setting I changed from default was to enable "USB Emulation".  I had to do that to get the thumb drive to show under boot devices.  I rebooted the system and it's still running fine tonight with all drives spun down.  Once again, it's been up ~24 hours, but it did this before, so I'll reserve judgement until it's been a couple days.

 

RobJ - You're right, I haven't tried disabling drive spin down.  I got caught up in trying all these other things and I overlooked that suggestion.  If it crashes again I'll try that.  I have seen times when all the drives were spun down and it was still running fine.  I haven't been able to time when this happens because when I reboot after a lockup unRAID immediately starts to do a parity check which takes ~3 hours.  The timing is usually such that I get home in the evening to find unRAID frozen and by the time it finishes a parity check I'm either already in bed or going to bed.  I will try both you suggestions if I continue to have problems.

 

yss - My system:

 

Epox EP-8HMMI-A mobo

AMD Sempron 64 3000+

1 stick of 512MB DDR (Hynix)

1 - WD 500GB SATA (parity)

1 - WD 160GB IDE (disk1)

1 - Samsung 500GB SATA (disk2)

 

Thanks for all the replies.  I appreciate everyone taking the time to try and help.

Just a note: Depending on the speed of Parity, the 160g drive can go into not-spinning state, while parity is being calculated on the last part of the 500gig datadrive.

I have seen it happen as I have disks ranging from 80 to 500gig, and the small disk go not-spinning, while parity is still being calculated on the rest of the disks.

So if not-spinningg state is the cause, it can happen during parity.

 

Disclaimer: I can't destinguish between different not-spinning harddisk states, standby, sleep, spin-down.

 

/Rene

Just a note: Depending on the speed of Parity, the 160g drive can go into sleep mode, while parity is being calculated on the last part of the 500gig datadrive.

I have seen it happen as I have disks ranging from 80 to 500gig, and the small disk go into sleepmode, while parity is still being calculated on the rest of the disks.

So if sleepmode is the cause, it can happen during parity.

 

/Rene

 

By 'sleep mode' do you mean drive spin down or something different?

post edited, "sleep" substituted with "not-spinning" and disclaimer added.

/Rene

Well, it managed to go a few days this time before locking up.  I was out of town for a couple days and when I came back it was locked up again.  While I was away, my wife verified for me that it was still up and running, but it crashed sometime later.  I have now disabled drive spin down and we'll see what happens.  Frankly, I don't hold much hope that anything will be different, but hopefully I'll be wrong.