Hi All,

 

I've been using Unraid for quite some time now (2years?) I'm running the pro version and its been flawless on all releases for me, however as of late my unraid server will just randomly crash.

It took me a while to fiqure out whats going but if i put load on the machine the following happens, (please excuse the photos they are the only way to show the output as its not logged to the syslog. Also attached are 2 syslogs.

 

Can any body help here I'm kinda lost at where to start..

 

Thanks,

Synth

flashlog1.txt

Screen shot 1

screenshot 2

If these crashes don't seem to coincide with any recent updates/changes to hardware or software, I would first check for signs of capacitor plague on the motherboard and power supply, then if everything looks good, run a memtest.

 

Sounds like a failing power supply.

 

Thanks I missed adding this machine passes a full 24hours of memtest, also have replaced the PSU with a known good unit (Silverstone 500w), At the moment it will crash while doing a parity check.

 

I can also confirm that all CAPs on the board look pristine.

 

Any other things to check? basically my NAS is no longer a functioning unit

Hi there. I'd suspect that the USB drive could be the culprit when it's reading/writing to it. I'd do a back up of your current USB that use for your unRAID server, especially the .key file and other important files under the /config folder, do a complete full format of the USB drive, perhaps several times to see if it comes up with any errors, run a USB drive test to see if there is any errors also.

Once your certain that the USB drive is A OK, do a clean fresh install of the unRAID server, only restoring the .key file onto the drive, and reconstruct your unRAID server with the drive order and other settings. Do a burn in test for 48 to 72 hours on your unRAID and see how that goes. I'd be interested on how you went. It could be a mainboard or USB port issue, but concentrate one thing at a time. Good Luck!

1.  How many drives and which controllers

2.  Which model Silverstone PSU?  How many 12volt rails does it have?

 

Have you checked CPU temps?  Perhaps it is a thermal issue.  Try a CPU stressing program.

 

I would also remove and re-seat all cards, RAM, and power connectors to the mobo.

Thanks SMNAS thats a good process to run though which i'll start now and report back with how it goes.

 

BubbaQ: 10drives (some 4xIDE + 6x Sata) all on MSI K8N Diamond plus mobo (using onboard controllers, one Nforce controller the other a SIL unit, All been running very successfully for over 1 year)

 

PSU = silverstone ST50F (2x 18amp 12volt lines) prior was a Server grade NMB 460w with the same 12volt rails.

 

 

Thanks Synth

No worries mate, bubbaQ also made some valid points to investigate too, hope you sort out the issue , Cheers!

SMNAS: I have formated (not quick) my USB key 5times, prepped it and installed a fresh copy of unraid 4.5.1 on to it, then my KEY.

Started the server, reconfigured everything and then started the parity rebuild it got to about 3-4% and did what is in the screen shot

Any further pointers would be great.......

 

Thanks

Synth

I'm only a newcomer to unRAID with just a basic server.  Is it possible something on the motherboard that's deteriorating is overheating?  Northbridge maybe?

I'd have to agree, it seems the Linux Kernel for unRAID isn't liking your mobo/CPU or RAM. It is obvious that the USB drive isn't at fault. Is your hardware old or fairly new new? I'd run (as suggested earlier) a stress test on your kit. Try re-seating the hardware as mentioned earlier, as a slightly disconnected component can cause a mountain of trouble and only take 5 minutes to disconnect and re-connect/re-seat everything again, practically anything that you can physically plug in and out.

If that doesn't do the trick, I'd do a full memory test (If not already done) to determine if the RAM module(s) could be dodgy, (unRAID has a built-in Memory Test at boot up). If a full RAM test comes up with no faults, the next up is to stress test the mobo/CPU parts. I'd recommend you run a Linux-based utility called StressLinux, which can do full tests on the mobo components, CPU, RAM, etc... Run this test for no less then a day or two, non-stop and see if this reports any problems on your hardware. Then, if your still running into trouble and if you have spare hardware that your current unRAID server uses, try swapping them one by one.

unRAID requires small computing power, but a parity check would make the CPU and mobo components work harder. If your kit is failing doing a parity check, by which it is doing a fair bit of calculating, the fault sounds to be a mobo, RAM or CPU specific one.

The unRAID OS constantly writes to the USB drive which its installed on and if their is corrupt portion of the USB drive, it would be enough to make the OS crash, as it can't read/write to that area. That was the reason I immediacy suspected the USB drive, as I've came across a few dodgy ones, even fresh out of the packaging.

You could disable any unused integrated hardware which might be triggering the problem, devices like Sound, Serial, Parallel, etc... Try a BIOS factory reset. Also, sounds like a obvious suggestion and a bit of cliche idea, but have you done a BIOS update? A simple suggestion but could make a world of difference in some cases too. Sounds like you've got some further investigating to do. Keep us all posted, I'm interested on what the fault actually is. Cheers!

Awesome reply  SMNAS and thanks Queeg, I'll run over this again for you guys:

 

Machine passes memtest consistently (left running for approx 20 iterations) seeing as this is a Athlon64 based machine memory controller is in the CPU.

Hardware is "old" socket 939 kit however I have a spare CPU which i'll swap out I also have another different socket 939 mobo which I can swap out.

 

To answer you questions further EVERY integrated peripheral in bios (floppy controller, serial, parallel, firewire, ac97.....) has been disabled from the first day I started to run unraid.

 

So now i'm going to start the laborious task of swapping out the hardware! I'll report back tomorrow once i've swapped out the CPU and memory.

 

Thanks,

Synth

No worries mate, it is pointless having knowledge if you don't spread it around . Hope it all works out for you and yeah keep us posted.

yeah funny thing is I had already started down the problem solving track of replacing hardware, but was hoping somebody would be able to tell exactly what was happening me being a windows person not a *nix person.....

 

I've removed the Ram and replaced it with a single dimm, doing a parity check now... time for bed.. will report back tommorow with an update.

nope, fail with different RAM, now time to replace the CPU

It's a good thing you have the parts to test out, one part at a time . Just by curiously, the machine isn't overclocked is it?

nope, fail with different RAM, now time to replace the CPU

 

If the replacement CPU fixes the issue, but has a lower current draw than the original, it could point to the motherboard being the issue.

 

Sorry didn't get a chance to change out the CPU, currently the CPU is a athlon64 3200+, I had it underclocked to 1Ghz (from 2ghz), but have tried it at factory settings.

 

The CPU I'm going to put in a dual core so would be very surprised if it draw any less than the current cpu....

 

Cheers,

so just fired up the machine with the Athlon 64 X2 4200+ cpu started the parity and it fell over VERY quickly, I'm going to make the "leap" and say this is the mother board causing the problem. Even so I've pulled the motherboard and put it on the bench and will  physically check over the entire thing then run it through some stress tests.

 

Will update with further info

 

p.s. my "replacement" board doesn't have enough SATA ports to implement my array so there goes that idea

Ah geez, the mobo looks to be the problem. Ah well, so long you've isolated the faults by doing a divide and conquer (which it seems you have), the only thing that you haven't replaced it that. Fingers crossed mate, hope you get the unRAID doing full parity checks soon  .

hrrrmmm the plot thickens, I didn't buy that my mobo was completly dead yet so on the bench I added 2 sata drivers and just used the free version of unraid, started parity -> crash, stuff around with the NICs and changed to the onboard=no diff, then i moved my disks off the SIL3132 controller (onboard) onto the native Sata controller = parity works, so i moved it back to the SIL controller = just about instant FAIL.

 

I've tried 4.5.3 and it make no difference, so i'm going to attempt using 4.4.2 and see what happens....

 

Cheers,

Synth

I can confirm that regardless of version,if there is a sata disk connected to the onboard Silicon Image controller Unraid will crash/Kernel Panic.

 

Just to reiterate this config has been running successfully for nearly two years so I'm reaching but I'd say something to do with the controller has bit the dust.

 

If anybody has any other ideas please let me know, otherwise i'm gonna have to find a PCI-e sata controller that doesn't cost the earth!.

 

Cheers,

Wow so the SI Interface ports causing ya greif? It seems the on-board SATA controllers for your mobo for the SI interfaces are stuffed. A bit if a rare problem, but stranger this have happened before. To give it a nw leave of life, just by an add-on SATA controller, but if your worried the rest of the board will fail (which it could), I'd simply replace the whole mobo, to be sure to be sure.

Cheers SMNAS yeah thats pretty much where I'm at now, I'm looking at modifing the system bios so that I can run the latest SIL Bios for the adapter and see if that gets me any where but at this point looks like i'll be without my NAS for another week