Parity Drive Failing & Replace Procedure

[trurl]

You could do it now, but if you shutdown you would have to do the New Config again since it doesn't record your disk assignments until the array is started.

 

So, you might as well wait until after you sync parity.

 

Not sure if I already said it in this thread (I have several going as usual) but when you do remove them, you should always double check all connections whenever you are mucking about in the case. Disturbing a connection is a very easy way to start throwing I/O errors.

[someoneotherthanyou]

Roger that. I’ll check back in after the parity rebuild and let you guys know the results. Maybe a dumb question, but should I refrain from using the array before parity rebuild? 

 

 

[trurl]

It will be a lot slower if you use the array much but it won't actually break anything.

 

Reviewing the thread I remember we wanted to try to get SMART on that missing disk6 to see if it is worth keeping. Is it connected?

[someoneotherthanyou]

I elected to remove the drive PRIOR

3 hours ago, trurl said:

You could do it now, but if you shutdown you would have to do the New Config again since it doesn't record your disk assignments until the array is started.

So I decided to shut down the server and remove the suspect 8TB (parity) drive. Did not touch the 4TB (failed) drive.

3 hours ago, trurl said:

So, you might as well wait until after you sync parity.

Really, really wish I would have done that as I am now faced with this upon restarting it. How freaking FUBARD am I now?

I had to unplug a total of 5 drives in order to ascertain which drive was the parity drive. It occurs to me now that which SATA/SAS port the drives are connected to might be somewhat important (ya think?). Is that my issue here?

 

[someoneotherthanyou]

After an hour of slight dismay and mildly freaking out I actually applied a bit of logic and reassigned the parity disk to the old disk7 (8TB) as well as properly reassigning the remaining disks to their proper slots and fired up the array. 

 

I sighed a huge sigh of relief when presented with this on the server

 

And this on my Windows 10 system

 

This has been quite an adventure for sure. I've taken the original 8TB parity drive and put it into my external USB 3.0 SATA dock to run diags on it. Crystal Disk failed it as did WD Data Lifeguard. Very disappointing as the drive only had 2255 hours on it. It began it's life as an EasyShare external drive with 3 of it's brethren for the beginning of my Unraid journey. Fortunately the drive is covered under warranty until 8/2020 so it will get sent back. As to the mysterious disk6? It is a wicked ancient 4TB drive I had laying around but at some point I will pull it from the server case and test it. For now it just sits unassigned. It's a pretty safe assumption it is a lost cause. No data on it though so I'm not overly concerned. All drives will fail eventually and it's time had come. 

 

Thanks again for all the help in getting me through this debacle. It is very much appreciated.

 

Best

[trurl]

3 hours ago, someoneotherthanyou said:

It occurs to me now that which SATA/SAS port the drives are connected to might be somewhat important (ya think?)

Unraid keeps track of disk assignments by disk serial number, so for that purpose the ports are irrelevant.

1 hour ago, someoneotherthanyou said:

I actually applied a bit of logic

Like this?

On 12/9/2018 at 8:29 PM, trurl said:

But for now, make careful note of each of the disk serial numbers. The screenshot you already posted is a good record, or you can print out that page. And we can also see them in the diagnostics you already posted. So they aren't in any danger of getting lost. But you will need to know them yourself when you complete the next steps.

 

1 hour ago, someoneotherthanyou said:

Thanks again for all the help in getting me through this debacle. It is very much appreciated.

👍

 

Going back into the thread a bit for some additional advice:

On 12/9/2018 at 1:48 PM, someoneotherthanyou said:

I started getting notices the other day that the 4TB and the 8TB parity drives were throwing errors.

Does this mean you DO have Notifications setup to alert you immediately by email or other agent when Unraid detects a problem? People who don't get notified when they get their first problem often end up with multiple problems that are more difficult to recover from.

 

And this:

13 hours ago, someoneotherthanyou said:

I guess my ultimate question is how much danger am I in of losing everything on this array?

Do you have backups? Parity is NOT a substitute for backups. You don't have to backup everything, but you absolutely must have another copy of anything important and irreplaceable on another system.

[someoneotherthanyou]

9 minutes ago, trurl said:

Like this?

Yup, exactly

 

10 minutes ago, trurl said:

Does this mean you DO have Notifications setup to alert you immediately by email or other agent when Unraid detects a problem?

Absolutely I do....My inbox was going crazy!

10 minutes ago, trurl said:

Do you have backups?

Repeat after me...RAID is NOT a backup...RAID is DEFINITELY not a backup.

I had just setup a docker for my CrashPlan Pro account prior to this. That was the first task I started once the array came back up. 

[someoneotherthanyou]

Closing the loop here and final update. Parity has been completely rebuilt, array is happy, and I owe you all a beer!!!

 

Thank you again and Happy Holidays to all!