I'm running 4.4.2 and suddenly found that one drive was listed as missing when in fact it was there. I restarted and it now had a blue dot, as if it was a new drive. Thinking I had nothing to lose, I started to reconstruct the data, but after a few minutes (less than five) the array stopped (no drive access lights showing on the front - definitely if parity is being accessed, my case shows a red drive access light). I couldn't telnet in, no response from the console, no web GUI.

 

When I restarted, the reconstruction resumed but stopped again after a few minutes. This happened a couple of times more. Thinking it was a drive problem, I changed the drive for another one (400GB IDE, same as old one), but go the same problem. I then tried swopping the drive cable, which was attached to a PCI card to the other PCI card, which had a spare IDE port, but again I encountered the same problem.

 

So I'm beginning to think it's not a hardware problem (not the drive or the PCI card) but a software one, but I can't think of any way of solving it. (Will try changing the cable later - it's a standard flat one.)

Changed the cable as well, but still having the same problem. One thing I also noticed is that the red light on my server blinks only intermittently during the reconstruction before it stops blinking entirely. In the past during reconstruction I'm pretty sure the light was on almost continuously. This indicates that the parity drive is being accessed only intermittently during initial reconstruction. I would have thought it would be accessed continuously during reconstruction?

There is absolutely no way for anybody to diagnose this remotely with you just describing how the disks blink their status lights.

 

To troubleshoot, since the machine is locking up, you must first attempt to gather details of the failure.

 

Log in via telnet, then type:

tail -f /var/log/syslog

 

Wait for the re-construction to stop and then take a picture of the failure on the screen at that point (with luck it might be there)

 

In the interim, crashes are most often caused by HARDWARE... so the very first thing to do is to check your memory.  Reboot and choose the memtest option.  Let it run for a few cycles or even overnight.

 

Verify the memory voltage, timing, and clock speed are all set properly in your BIOS for your specific model/brand of memory strips.  Some BIOS get it right automatically, some do not and have to be set manually.

 

Post a syslog, if you can.

 

Whatever you do, do not press the button labeled "Restore" as it does not restore data, but stores a new config based on the current working disks.  If you have a real disk failure, it would throw away any parity that would allow you to reconstruct that disk.

 

Joe L.

Ah, apologies for not adding diagnostic information. I thought that the only option was getting the syslog, which wasn't possible because connection with the server was lost once the error occurred.

 

I ran a memtest; just 4 passes, but things looked OK. Then started up the array and the resulting output is in the attachment.

This is only a small part of the whole syslog, but there are some clues here.  You mentioned above about an IDE drive and changing a flat cable, but the only drive mentioned in this syslog piece is sda, a SATA drive, with serious communications problems.  The problem looks like a bad SATA cable to sda, but it could also be a bad power cable, or a power problem to this drive.  Now if this is not the drive that showed as Missing, then there may be problems with 2 drives.  But this drive (sda) is probably the one that is stopping any reconstruction.

 

If you were able to obtain this tail piece of the syslog, you should be able to capture the syslog at the same time, once you see these exception Emask errors starting to occur.

Rob, thanks for the reply. It's entirely possible that I knocked something loose while fiddling inside the rat-tail nest inside my PC. I'll see if I can do something about it this weekend.

I restarted the system and ran into the same problems, but this time I captured the syslog up to the point where communication was lost.

Thank you for posting the entire syslog.  Unfortunately the errors at the tail end of this syslog are identical to the previous megabyte of syslog errors!  There is no other issue mentioned at the end, to definitively say what actually is stopping your server.  The intermittent blinking is completely consistent with the errors reported, as the constant resetting of the drive involves constant pauses of 5 seconds, during which nothing else can happen.  At this rate, the drive reconstruction would not be measured in hours or even days, but weeks and perhaps months!

 

I can say that, although I did not examine all of the over 15000 lines, that ONLY one drive is causing all of the trouble, sda, and that it still looks like my advice above is correct.  I would not do anything with this server until you replace cable(s) to this drive.

 

From your first post, it is possible that the original Disk 7 is fine.  You might try re-installing it, and Obtain a SMART report for it.  If the drive appears fine, you may want to use the Trust My Array procedure.