Jump to content

Large unRaid Server, Lightning strike, now fails to POST. Please help.


Juano11

Recommended Posts

I've got a large unraid server:

 

Supermicro X7SBE MoBo

Intel E6500

G.SKILL 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333

CORSAIR CMPSU-750TX 750W ATX12V

SanDisk Cruzer Micro 2GB Flash Drive

SuperMicro SASLP-MV8 SATA cards (2)

(8) WD Green 2TB drives - One Parity, 7 data drives

A smaller Cache drive that I don't remember what it is

Norco 4020 case

 

This was behind a CyberPower PR1500 UPS along with my modem and router.

 

Well...the house got struck by lightning last week and fried quite a few things throughout the house, including my modem and router.  Initially, all seemed well with my server.  I connected a new router and I believe that I was able to connect to the unRaid again (although, in retrospect, I'm not sure that I was able to do this).  Anyway, after getting my network and internet connectivity sorted out, I realized last night that I could no longer connect to my server.  I tried re-booting all of my equipment and still nothing.

 

After restarting the server a few times with the power button, I realized that I was not getting the POST beep.  Fans and LEDs run, but no POST.  Additionally, after connecting a monitor, I found that I was getting no video output.  No MoBo splash screen...no error message...no unRaid...nothing...monitor not getting a signal.

 

So, something got zapped.  It's strange to me that it seemed to work for a few days after the strike and now does not, but oh well...who knows?

 

I'm writing to look for advice both on how to troubleshoot my server hardware AND how to preserve the data on my drives.

 

 

First, any advice about how to go about trouble shooting the hardware? 

 

    Should I try to reset the CMOS and see what happens? 

 

    Where to start when swapping out new hardware?  PSU first?  Memory?  MoBo?

 

Second, assuming that my data is still intact on the drives, what should I do to preserve it while I'm troubleshooting.

 

    Disconnect all the HDDs while troubleshooting?

 

    Only disconnect the data drives?

 

    Disconnect the SATA cards?

 

 

Finally, assuming that I can locate the offending piece of hardware, how to I go about re-connecting my HDDs to ensure that I do not lose data that is already on the drives?

 

I love my server and have about 10TB of data on there that I desperately want to preserve.

 

Any and all help would be greatly appreciated.

 

Many thanks

 

John

 

 

 

Link to comment

I would start with pulling the plug on the power supply from the wall or UPS.

Then check if the motherboard lights come on.

Pull the flash ret and boot.

Then reset the CMOS, try and boot.

Then memory, just because it's the easiest thing to do.

Swap the video card if you have one.

 

I would pull the controllers while they are still plugged into the drives making sure to take note of the slots they were in.

 

However, looking that you have a norco case, I might pull each drive out slightly so that it does not touch the back plane first. One drive can hang a motherboard.

 

Then pull the controllers.

 

I've heard that a fried backplane can cause the machine to fail.

 

By the time you get here, you are close to pulling the machine apart.

 

 

I've had a machine fry on me.

What survived was the CPU, fan, and hard drives. Everything else was dead.

Link to comment

Thanks for the reply.

 

The motherboard lights do come on.

 

I have not tried booting without the flash or resetting the CMOS.

 

I have been hesitant to do much because I, unfortunately, only know the drive assignments for my parity, cache, and 3 of my 7 data drives.

 

What steps, if any, can take to ensure that I do not lose the data on my drives?  Assuming the data is still intact on the drives, I don't want to "fix" the unRaid box to find that my drives are overwritten when I re-install them. 

 

I believe that all of my data drives are connected via the SATA add-on cards.  I'd prefer, if possible not to mess with the intenal SATA connections on the backplane because they are rather fragile (I've accidentally pulled the connector from the pins in the past).

 

Thanks for the reply.

 

I just want to make sure that I'm not walking into a pitfall here with my data.

Link to comment

If the drives come up in the wrong slots, the array will not start.

if more then one drive is missing, the array will not start.

 

If the sata connectors on the backplane are fragile, leave that for last.

However if you have removable sata units, you can pop the drive out halfway from the front when you get to that step.

 

It probably should be something that unRAID does where it documents the drives and slots.

I suppose you could copy the flash to another flash drive, boot that somewhere else.

unRAID cannot start the array, but you will at least be able to see what drives and serials are in what slot.

But as I mention before,. if any drives do not come up the array will not come up and will not write to any drive.

 

Link to comment

OK, still no love.

 

I've tried rebooting with each of the drives disconnected, no change.

 

I've tried rebooting with the drives disconnected from the SATA cards, no change.

 

I've tried rebooting without the flash drive, no change.

 

I've tried swapping out the two memory sticks, both the sticks themselves and all possible combination of slots & sticks, no change.

 

I'm actually not sure how to reset the CMOS, so I haven't done that yet.

 

 

 

If I take my data drives out, can a windows PC see them?  If I have to, I'll buy the discs to copy and save my data.

 

What will happen if I need anew MoBo and don't know my drive numbers?

 

I've copied my flash drive to another PC.  Can you tell me where to look for my drive assignments?

 

As I'm doing this, should I keep my data drives disconnected?  I don't want to accidentally overwrite them.

 

 

Ugh...If you can't tell, I desperate for help.  I don't want to do something, only to find out that you can't recover your data after taking THAT PARTICULAR STEP.

Link to comment

I used to do PC tech support in the late 90's and one thing I hated most was when a lightning strike machine came in. They seemed to defy logic. ???

 

I had one PC that would not boot (blank screen, like you're experiencing) until I left it on for no less than 10 minutes and then pressed CTRL-ALT-Del.

 

So, good luck!

Link to comment

Ok, I've gotten it all the way down to psu, mobo, memory, and processor. Nothing else connected. No HDDs, no cards, no USB stick.  I believe that I have cleared the CMOS.

 

Still no POST, still no video.

 

At this point, can I assume that the MoBo is bad?

 

If so, how can I preserve my data while replacing the MoBo?  As I mentioned above, I do not know all of the drive assignments. I have copied the USB stick to another computer.  Is my drive assignment located somewhere on there.

 

This lightning strike has really wreaked havoc at my house, I'd really appreciate help in recovering my unRaid and it's data, if possible.

 

I'll rebuild the whole damn thing if you guys can help me save my data.  Can I connect the data drives to a windows machine to copy?

Link to comment

 

I'm actually not sure how to reset the CMOS, so I haven't done that yet.

 

It is in the motherboard manual.  You can get is from http://www.supermicro.com/support/manuals/index.cfm

 

If I take my data drives out, can a windows PC see them?  If I have to, I'll buy the discs to copy and save my data.

There is a ReiserFS driver that will allow you to read the disk in windows.

 

What will happen if I need anew MoBo and don't know my drive numbers?

There are ways to get it back...

Link to comment

Ok, I've gotten it all the way down to psu, mobo, memory, and processor. Nothing else connected. No HDDs, no cards, no USB stick.  I believe that I have cleared the CMOS.

 

Still no POST, still no video.

 

At this point, can I assume that the MoBo is bad?

 

If so, how can I preserve my data while replacing the MoBo?  As I mentioned above, I do not know all of the drive assignments. I have copied the USB stick to another computer.  Is my drive assignment located somewhere on there.

 

This lightning strike has really wreaked havoc at my house, I'd really appreciate help in recovering my unRaid and it's data, if possible.

 

I'll rebuild the whole damn thing if you guys can help me save my data.  Can I connect the data drives to a windows machine to copy?

 

If not more than one drive is fried then you don't have to worry about losing any data. You can rebuild your server with different parts and then reinstall the drives. Assign them all as data drives. A single drive should show up as unformatted. The unformatted drive is your parity drive. Assign the parity drive to the parity slot and start the array. UnRAID does not care much about hardware. If your drives are OK then you can easily reconstruct. Share should appear with default settings. You may have to reconfigure the details of your shares if the config directory of your flash drive is lost.

Link to comment

my last power surge (possibly lightning) based pc disaster was the power supply.  I could turn it on the fan in the PSU would spin the lights would come on but the machine wouldn't post.  Hooked up a new power supply and that machine worked until I replaced it.  I'd very much second Joe's thoughts on the PSU at this point.

Link to comment

go with a new PSU for now.

 

If you really had to, you can boot the unRAID flash on another system.

It will come up with no drives. But at the MAIN & DEVICES screen you can see what drives are configured.

At that point you can identify the parity drive.

Once you identify the parity drive, all others are data drives.

 

Data drives can be remounted on another unRAID machine "without parity" to let you access the data over the network.

You can also access the data with a reiserfs driver for windows.

Link to comment

If you know the parity and the cache disk then you are fine. You won't lose any data if the disks survived. You can start over completely and put the data drives in any order and you can re-create the array without data loss. About the worst that happens if you re-arrange disks is that the share settings assigning user shares to certain disks gets messed-up (eg, the assigned disk5 doesn't hold that share anymore).

 

Just worry about getting the server operating so the disks slots in the case all have good connections again. It might not be a bad idea to use some old smaller drives (still good though) and test each slot, possibly running part of a pre-clear or some other boot-disk or boot-USB utility to confirm the controllers are still OK.

 

When you first boot, the main unRAID screen should tell you what disks are expected where so if you do get that screen do a screen capture to keep that data.

 

Peter

Link to comment

OK gentlemen, installed the new PSU and tried to boot with nothing connected.  No HDDs, No USB, No SATA cards.

 

No Joy.

 

Still did not beep for POST, did not yield any video output, I even tried a second monitor.  Fans spin, MoBo LEDs light up as if everything is A-OK, but nothing happens.

 

I again tried both memory sticks in all possible configurations with no results.

 

So, I guess that I have to assume that my Motherboard is bad.  Correct?

 

 

Link to comment
  • 3 weeks later...

Yep, it was the motherboard.  Got an identical one online for ~$230.

 

Hooked everything back as it was originally and it all came back as though nothing had happened.

 

No drives damaged, no data lost. Excellent.

 

It cost about $500 to get it figured out/fixed, but now I've got an extra PSU laying around that will make my next build that much cheaper.

 

The Supermicro MB is more than a year old, so it is out of warranty (the damage was probably not covered anyway).  Has anyone tried to return a board to them for service?  I think that I'm going to try and see how much it would be.  If it's ~$100 to get it fixed back up. I might do it.  That way, I'd have a PSU and MB on the way to another big unRaid server for a buddy on the cheap(ish).

 

What do you think?  Is it worth trying to get this motherboard fixed?  I know that there are lots of unknowns, but I've never tried to get a motherboard repaired.  Does it usually cost more than it's worth?

Link to comment

Yep, it was the motherboard.  Got an identical one online for ~$230.

 

Hooked everything back as it was originally and it all came back as though nothing had happened.

 

No drives damaged, no data lost. Excellent.

 

It cost about $500 to get it figured out/fixed, but now I've got an extra PSU laying around that will make my next build that much cheaper.

 

The Supermicro MB is more than a year old, so it is out of warranty (the damage was probably not covered anyway).  Has anyone tried to return a board to them for service?  I think that I'm going to try and see how much it would be.  If it's ~$100 to get it fixed back up. I might do it.  That way, I'd have a PSU and MB on the way to another big unRaid server for a buddy on the cheap(ish).

 

What do you think?  Is it worth trying to get this motherboard fixed?  I know that there are lots of unknowns, but I've never tried to get a motherboard repaired.  Does it usually cost more than it's worth?

Many times homeowners insurance will cover damage due to a lightning strike.  If you have that type of policy, you might want to check it out.

 

About any repair center will do is replace the motherboard with one that is refurbished... something that was zapped by lightning is not likely to be repairable... It is not as if they can unplug components and try substitutes, and any un-soldering of components and diagnosis would take so much time as to make it economically unfeasible to repair.  Even if there was obvious damage to a specific part, there is no way to know if replacing it will get the board to boot once more.

 

Keep an eye out on e-bay for a similar MB and use it for your next server.  The great part of unRAID is it really does not care about the specific motherboard...  and for what you spent on the replacement ($230) there are tons of alternatives, some potentially more powerful since they are newer/faster.

Link to comment

Good to hear that you got it back up and running without any data loss!

 

Yep, it was the motherboard.  Got an identical one online for ~$230.

 

Hooked everything back as it was originally and it all came back as though nothing had happened.

 

No drives damaged, no data lost. Excellent.

 

It cost about $500 to get it figured out/fixed, but now I've got an extra PSU laying around that will make my next build that much cheaper.

 

The Supermicro MB is more than a year old, so it is out of warranty (the damage was probably not covered anyway).  Has anyone tried to return a board to them for service?  I think that I'm going to try and see how much it would be.  If it's ~$100 to get it fixed back up. I might do it.  That way, I'd have a PSU and MB on the way to another big unRaid server for a buddy on the cheap(ish).

 

What do you think?  Is it worth trying to get this motherboard fixed?  I know that there are lots of unknowns, but I've never tried to get a motherboard repaired.  Does it usually cost more than it's worth?

Link to comment

Yes, my insurance would cover it, but I have a $1000 deductible.  Luckily a TV got fixed by the manufacturer (big props to Costco & Vizio, they even came to my house!) and our garage door opener got fixed by our home warranty.  The remaining damage, including the $230 for the server is probably only about $2000.  I don't think that the $1000 that I would get from insurance would be worth having a claim against the house.  So, oh well, I'll eat it.

 

I considered trying to make a claim with the UPS manufacturer, but the UPS is a good 7yrs old, I doubt that they would still cover it.  In any case, I can't find the paperwork/receipt for the UPS.  I'm sure that I'd need that to make a claim.

 

I'm not going to mess with it at this point, but has anyone ever tried to make a claim against a UPS/Surge Protector connected equipment warranty.  I always figured that they would make it very difficult and almost always find a way to deny it.  Anyone with any experience?

 

Thanks again for the advice and reassurance.  For a little while there, I thought that my shit had gotten wrecked....

 

John

Link to comment

It would be interesting to see how the UPS manufacture would respond.

I agree that it's probably not worth the time and effort. But still, sometimes they advertised such outrageous claims of protection, I'm surprised the motherboard fried while behind a UPS.

 

Makes me want to have some form of kill switch to disengage all my computers and keyboards and other musical electronics should a heavy thunderstorm occur.

Link to comment

It would be interesting to see how the UPS manufacture would respond.

I agree that it's probably not worth the time and effort. But still, sometimes they advertised such outrageous claims of protection, I'm surprised the motherboard fried while behind a UPS.

 

My cable modem & router were also connected to the UPS and got fried.  The modem & router were sitting on top of my server case.  the modem has/had little round vent holes in the top & bottom of it's housing.  I wish that I had taken a picture, but the server case had about 10-12 little, perfectly round burn/singe marks on it from the modem frying.  Additionally, I'm fairly certain that my server died a day or two after the strike...Weird.

 

Interestingly, it did not fry my big D-Link switch or very old HP monitor that were plugged into the same UPS.

 

Also, the UPS still seems to work fine.  I've bought a newer APC unit for my server, but re-purposed the old one as a battery backup to my basement waterproofing pump.

 

All in all very strange with regards to what got zapped and what didn't.

 

Oh well....

Link to comment

This makes more sense now.

The lightning probably arced from the cable modem to the server or vica versa. Either one of them had a different ground potential and there was an arc across them. I guess for the future separate them and/or get some kind of cable isolation power strip.

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...