Hello!

5days ago I upgraded one of my 320GB drive with a 500GB.

Everything was fine until this morning when I had to hard reboot the unRAID box as it was not responding anymore after a nightly parity check.

Upon reboot data seemed fine until I checked the array and noticed the new (disk3) drive with a red status.

Quick look in the syslog (attached) doesn't indicate any error.

Strangest thing is I can access the content of the drive! Via smb and via telnet.

I tried to unassign and reassign the drive with no significant change.

 

Any idea on how I can fix this and restore a protected array?

 

I'm running 4.2.3 , any help would be much appreciated

If a drive fails, unRAID simulates the failed drive by reconstructing data from the various other drives in the array together with the parity drive.  It is no surprise that your array would continue to operate even with one failed drive.  I am not an expert at reading the syslogs - hopefully someone more knowledgable will be able to pick up on something and let you know.  If the drive has indeed failed, you should be able to replace it with a drive of equal or greater capacity and unRAID willreconstruct the failed disk.  I'd hold off, though, until you get more data.

 

Your syslog does look fine.  There's only one odd line, "read_attr: fopen: No such file or directory" immediately following the disk 'Device inventory', but it appears to have been considered harmless, as it does not trigger any error handling, or result in any failures or apparent loss of any functionality.

 

To clarify, if you reboot now, does a red ball still show for Disk 3, and the resulting syslog is essentially identical to the one you attached (except for times)?

 

The array was restarted on

Apr 12 11:56:54 Alpha syslogd 1.4.1: restart.

 

And later when it tried to resync:

Apr 12 11:56:58 Alpha kernel: [  119.744364] md: recovery thread has nothing to resync

 

Hence a write error to your new disk occurred prior to the start of this syslog.

You can re-enable the hard drive and try to reconstruct it as follows:

 

1. Stop the array, go to the Devices page and unassign the disk.

2. Reboot

3. Stop the array, go to the Devices page and re-assign the disk

4. Go to Main - system should detect a "new" drive for the one disabled, click Start to start parity-reconstruct of the disk.

Look in this thread ... http://lime-technology.com/forum/index.php?topic=1521

 

for a file called smartctl.zip.  Follow the instructions to install (basically just unzip it onto your USB stick), and run it on your drive from a telnet prompt or firon the unRAID console.

 

It should give you more info about the drive's view of what has happened.  You might want to post the results here to get some other eyes to help you interpret its results.

 

If the drive truly experienced a write failure, that is a bad thing, and the drive should probably be exchanged for a new one (you said it was new)  It is not uncommon for a new drive to fail within the first few weeks of being put in service.

Hi there!

Thanks a lot for all the answers!

First of all, the error stays after a reboot. The new syslog looks pretty much the same.

Following your advises I installed and ran smartctl on the "failed" drive and attached the result in this post.

I'm no expert but this disk seems awfully fine to me with the exception of the 2 UDMA CRC errors... Could this be the issue here?

 

I'm going to try Tom's procedure and reconstruct the drive. Hopefully this will solve the problem. I'll keep you posted

 

EDIT >> Okay... Data rebuild will take about 5 hours 

I have 12 drives in my array.  8 Terabytes.  None of them have any UDMA CRC Errors.

 

I know what a CRC error is, but not sure what a "UDMA CRC Error" is exactly, but since whatever it is caused unRAID to kick the drive, I would exchange it for a new one if I could.

 

Western Digital has some disk diagnostic software - I think it is callled LifeGuard.  You might try running it on your drive to see if it can somehow exclude the problematic part of the drive.  You might also try Spinrite if you have it.

I believe UDMA CRC Error are communication errors. I.E. Cable.

I took a look at the smart log nothing jumped out.

You can run two self tests and dump the smart log again.

 

 

smartctl -d ata -tshort /dev/sd?

where /dev/sd? is the drive you are testing.

The short test takes 2 minutes

You can do a long test for good measure which takes about 2 hours

smartctl -d ata -tlong /dev/sd?

I did some Googling and confirmed WeeboTech's finding.

 

Possible causes of UDMA CRC errors are bad interface cables or cable routing problems through electrically noisy environments (e.g., cables are too close to the power supply).

 

In another thread I found a person that was getting these errors under Linux but not under Windows, and there was some motherboard specific / driver specific suggestions. But this person had these errors on all his drives.

 

Why don't you try changing the SATA cable to this drive?  (My gut still says to exchange to drive, but maybe that is not necessary)

 

(WeeboTech - I added your instructions for running these two tests to the wiki)

 

Well, reconstruction completed correctly, the drive is showing up green again.

Now I'm running a parity check as I had to hard reboot again due to server not responding. Yes it does that a lot after working for a while doing a p-check for example... But I don't have enough relevant info (syslog) to pinpoint anything and open a post about it.

Anyway, the short test completed, could you please tell me how can I find the log?

If you run the normal smartctl command again, you'll see the results of the test.  I just did it on one of my dirves (had never done this before), and it showed a line where the test had run without any errors.

 

I think your next step is to crack open the box, check the connections to this drive, and (I would) change the data cable.  If your computer is hanging running parity checks, you still have a problem.

OK, I decided to cancel the parity check open the box and look at those cables :

Those drives are in a 4in3 icy-dock box and although I saw nothing wrong with the cables I replaced them with a couple of new ones. I also rearranged them to be as far as possible from the power cables.

I reran the 2 tests and attached the result to this post.

 

Now I'll relauch the parity check

Again, thank you very much for your help troubleshooting this!

Tom, thank you for the procedure to restore my protected array

Good luck!