March 9, 201412 yr Hi All, I have been experiencing some weird behaviour when booting my unRaid server. I'm running 5.05. When I boot up the server, I can initially connect to it via telnet and through a web browser, however, shortly after this I will lose the connection and not be able to get it back. This means that I need to forcibly power down the server as I have no way of doing it in a safe manner (I tried blind logging in with a keyboard but couldn't get it to work). However, after doing a force restart / shutdown and startup again, it will boot and have no problems. Unfortunately this method does mean that the server does a parity check every time I start it up. And obviously isn't ideal. I have checked the system log and am getting some Namespace related issue but am not sure what I need to do to solve this. Attached is a full syslog and a truncated one just showing errors, warnings etc... I would be grateful for any help you can give me. Cheers! Sareon SOLVED: Bad RAM, replacing this now provides stable startups. Thanks to all for the help. shortSysLog.txt syslog-2014-03-09.txt
March 9, 201412 yr Is your BIOS current? Check for an update. Then describe your hardware: What motherboard, BIOS, number and size of drives, and, the add-in cards. Do you have an add-in card that provides disk drive interfaces? (Like a SATA card?) If so, what make and model?
March 9, 201412 yr Author Hi, Thanks for the reply. I'm not sure about the BIOS version that I have. I shall check when I am back at home. My setup is: 400W Silverstone Stride 4GB Corsair DDR3 XMS3, PC3-10666 Intel Core i3 3220T, S 1155, Ivy Br Gigabyte GA-B75M-HD3, Intel B75, S No add-in cards. HDDs: 2x 4TB WD Red. (Parity & Array Disk) 1x 2TB WD Green. (Array Disk) 1x 1TB Hitachi disk from an external HDD. (Cache) Many thanks, Sareon
March 9, 201412 yr See my sig to disable all add-ons. Does the problem still exist with no add-ons running?
March 9, 201412 yr Author See my sig to disable all add-ons. Does the problem still exist with no add-ons running? It still happens with no add-ons unfortunately. I have attached the sys log. Thanks, Sareon syslog-2014-03-09.txt
March 9, 201412 yr Mar 9 18:49:27 Datageddon pms: Starting Plex... Mar 9 18:49:27 Datageddon su[2198]: Successful su for unraid-plex by root Mar 9 18:49:27 Datageddon su[2198]: + root:unraid-plex Add-ons are still loading.
March 9, 201412 yr Reboot your unRAID and select 'safe mode' at the startup screen...then post that syslog.
March 10, 201412 yr Author Ok, I rebooted using the adjustments from 'dgaschk's sig. And this doesn't load add ons. The log I previously posted was one where I had corrected the settings and rebooted. I hadn't realised that there was no way to access the log of an instance of the server once it had shutdown. So unfortunately due to not being able to access the computer via telnet or the web browser I wasn't able to get a syslog (the server was running headless). I have now attached a monitor to the server and am waiting to be able to re-produce the error. As it seems to only happen after the server has been running for a while and is then shutdown and rebooted. Thanks for your patience. I shall get back to you when I have another (useful!) syslog.
March 10, 201412 yr Author OK, so I believe i replicated the problem. But unfortunately I haven't been able to get a system log as the system completely crashed i.e. was unresponsive so I couldn't save the syslog. I had to force shutdown. Is there anyway I can make sure the syslog is written to a file so that even if the server crashes I am able to see the log? I believe this crash is what is happening all the time. It would explain why I couldn't manually shutdown the server by plugging in a keyboard. It seems to occur when I use telnet to connect to the server. The only feedback of what happened i was able to get was taking a picture of the monitor output: http://tinypic.com/r/2el9ze9/8 Any suggestions as to the next step? Cheers, Sareon
March 10, 201412 yr A few ideas, may or may not help... If you are overclocking, don't. Do a long memtest, overnight at least. Let's make sure it's not bad memory. Keep a tail running on the console: tail -f --lines=120 /var/log/syslog Add vga=6 to your syslinux.cfg file, to the append line. This will give you more info (60 lines instead of 25) in your console pic, for your next crash. Change append initrd=/bzroot to append initrd=/bzroot vga=6 Somewhere, there are simple instructions for constantly saving the syslog, but I cannot remember them. Hopefully, that could provide a syslog from the last minute of the server's life. That may not help though, if the crash is sudden.
March 11, 201412 yr Author Hi Rob, Thanks for the help. I wasn't over clocking. I did find issues with the memory test. Immediately came up with some errors after running it. I reseated the memory and still had errors. Tried the other slot and errors weren't present and tested this slot overnight. In the morning I moved the memory back over to the dodgy slot and the errors didn't appear this time. So i suspect it was maybe something blocking a contact that got wiped away by the other slot or something. Either-way, i'm running with the ram stick in the slot that didn't show any errors, so hopefully that will prove to be more reliable. I'll make the changes you mentioned too. I have noticed that the 'namespace' error i mentioned in the log is still there. Does anyone have any suggestions as to what may be causing this? I have noticed that I have difficulty being able to reliably connect to the AFP Time Machine drive. Seems to only work if i connect to the server with my mac after a reboot of my mac. Normally comes up with an error related to access. Anyway, thats a side issue that I can live with for now. Thanks for the help. Sareon
March 12, 201412 yr Again, first priority is the basic system. Did you check your BIOS for an update? Try and get a syslog as RobJ describes in his note two postings up. Lower priority are the Mac issues. My experience with unRAID and Mavericks is that it just runs by itself. In your KEYCHAIN, try removing the saved unRAID login and pswd.
March 14, 201412 yr Author Hi all, Thanks for the help so far and sorry for my slow replies. I am busy with work at the moment so having to do this in gaps between other things. I have now updated the BIOS to the latest version (said to have compatibility improvements), and made the adjustments to the console screen. I'll look into getting the system to record the syslog too. Shall update again if I get another crash. Thanks for the help, Cheers, Sareon
March 18, 201412 yr Author Ok, so it is still be crashing. I've attached a syslog but it doesn't seem to have recorded any information about the crash. It appears that the crash happened shortly after power-on as it is set to auto wake at 7:30am, and the last entry is before 7:31. I'm going to try running another memory test to make sure that isn't causing any issues still, and i'll run the box with a monitor attached for now i think so I can try and get more info about the crashes. When I get the chance i'll also grab screens of my bios settings to make sure that you don't think I have some strange settings going on. Does anyone else have any other suggestions as to what may be happening, or things i should take a look at? Many thanks, Sareon syslogtail_2014-03-18_11-16-07.txt
March 18, 201412 yr Author BIOS pictures below: http://i57.tinypic.com/9a4w1x.jpg http://i62.tinypic.com/2u46fpc.jpg http://i57.tinypic.com/2a7dj11.jpg http://i58.tinypic.com/11boisp.jpg http://i60.tinypic.com/k1e23d.jpg http://i58.tinypic.com/efqflt.jpg http://i61.tinypic.com/16h3r43.jpg http://tinypic.com/r/t7iqo0/8 http://tinypic.com/r/2i11qvn/8 http://tinypic.com/r/339ktic/8 http://tinypic.com/r/vmsmt2/8 Cheers, Sareon
March 18, 201412 yr Normally, ACPI Errors are quite harmless, but yours are occurring rather late in the process, are repeating (multiple calls), and are related to the SATA support, obviously quite relevant. Are you positive this is the latest BIOS? A system that crashes that quickly and predictably becomes a little easier to debug, as you know you can monitor the screen precisely when it happens. Watch for unusual messages, and if you see anything unusual, use a camera to record. I assume this is a sudden restart, working normally then screen goes blank then reboots? And it does this even in safe mode, no Plex starting? If no other cause is found, then you may need to start pulling cards and keep rebooting until you can boot without issue. I'd also be interested in how the latest memtest goes.
March 18, 201412 yr Author Hi RobJ, thanks for the reply. Yes this is definitely the latest BIOS, i'm running F5 which is the latest available on their website. To confirm what happens / when it happens: The crash only occurs if the server has been turned off for a more than a short period of time. I.e. if it has been off overnight, or just for an hour or so. It doesn't restart, it just throws a load of output to the screen and then hangs, completely unresponsive. The crash occurs normally around 5 mins after the server has turned on. The only way to get it to respond is through the reset or power switches. Telnet, keyboard input and the web page are all inaccessible / unresponsive. If I then turn the server back on shortly after turning it off, it will work reliably (by using either the power switch to completely shutdown and then turn on again, or the reset switch. Both work). Once it has turn back on like this I have had the server stay up for days reliably until I turn it off for the next extended period of time. I managed to get two crashes to occur today by leaving it powered off for an hour or two. The pictures below show the monitor output when it did. http://tinypic.com/r/2pobhjp/8 http://tinypic.com/r/287ks4h/8 I've previously tried disabling add ons and it still seemed to happen, but i'll try booting up using the safe mode selection in a couple of hours to see if it happens again. I ran a quick 1 cycle memtest earlier and it didn't seem to flag anything, but I shall run another overnight to see if anything pops up. Thanks again for the help, Cheers, Sareon
March 18, 201412 yr I really hope you find something wrong with a memory card, because otherwise it may be something more expensive like the motherboard or addon card. You at least have found a workaround, either never turn it off, or turn it on and wait for crash then reboot. A remote possibility, try a v6 version of UnRAID. It is 64bit and using a newer kernel, therefore will organize memory and IRQ's and other resources somewhat differently. It also should have a newer ACPI handling section, so who knows, might make a difference. I'd be curious to know if, after cooling for a few hours, turning it on and running memtest for 10 minutes then booting UnRAID, would make any difference. It should warm up the CPU, memory, and power supply, but not much else. Probably wouldn't prove much, because the other components would still be on, even if not used.
March 18, 201412 yr Author Yeah me too. I'll try stealing a stick of RAM from my windows desktop and putting that in to see if it fixes it or not. Hopefully if it is the motherboard I may be able to get Scan to send me a replacement because this has been happening since I got it.
March 18, 201412 yr Author Ok, and another crash. This time in safe mode. This log contains a few more of the kernel messages from just before the crash. The difference with this crash was that the sever still responded to keyboard input for a short amount of time. I was in the process of trying to copy the syslog to a file but it threw more kernel errors when I entered the command. I then tried navigating to a sub directory of the boot drive to see if the log had been saved or not, and this crashed it. I could still type commands but i didn't have a prompt marker. I was just typing on empty lines with commands doing nothing. I'm going to try swapping over a stick of ram now. sysLogCrash.txt
March 18, 201412 yr Something different to try, or at least eliminate as a suspect, remove/replace the Logitech items, including any explicit Logitech driver loading commands. Your syslog is showing a Logitech wireless device loading, which is not unusual, but what is very new to me is the use of hid_logitech_dj (with possible hidden dependencies). I've never seen it in an UnRAID syslog before, so let's make sure it's fully compatible. I cannot think of any reason it should not be, but ... If the machine without any Logitech stuff crashes just as fast and often, then it's not the problem.
March 18, 201412 yr Author I should have mentioned that the logitech stuff is me plugging my keyboard in after the crash to see if it responded or not. In the past this crash has occurred regardless of whether I have plugged my keyboard into it or not. Do you still think it is worth checking out? Thanks, Sareon
March 18, 201412 yr Do you still think it is worth checking out? At first, I would have said probably not, especially if you meant that you did not have the wireless dongle plugged in. But when I look at the chronology of events recorded in the syslogs, it almost looks like a possible correlation, some rather tight windows and almost too coincidental timings there. In all of the syslogs and syslog excerpts, one of the very last statements recorded is the final avahi statement immediately after all drives are mounted and it announces the array is started, the same whether in safe mode or in normal mode with Plex starting. Crashes appear to happen extremely soon after that. In the previous syslog tail, it's the very last line logged. In this syslog piece, it happens at 17:03:34, then 3 seconds later the network links up, then about 17 seconds later the Logitech driver is loaded. That's about 20 seconds for it to crash, you recognize it's crashing, then quickly connect or enable the wireless device (you are fast!). Does that sound correct to you? Then there are about 11 minutes where I assume you were trying to diagnose what worked and what didn't, then the keyboard is disconnected. Just 3 seconds later, we have a kernel Oops, but the system was still somewhat operational for almost a minute, then the keyboard is reconnected and the syslog terminates for good, I assume with a hard crash. Sound right? I cannot assume much of anything here. I cannot conclude it's the Logitech's fault, or it's not its fault, but it looks like it *may* be related, or it is forcing the issue somehow.
Archived
This topic is now archived and is closed to further replies.