Replacing multiple failed drives

[anhlon]

Hi,

 

I am a happy user of unRAID for many years.  Recently, my drives starts failing all at once :(  Currently, I have 1 2TB parity drive, 3x 2TB data drive and 1 1TB data drive.

At the moment, unRAID disable my parity drive and I cannot enable it, the drive doesn't seem to have any errors from what I can tell.  There are 2 2TB data drives that are reporting read errors, I suspect that this is causing the parity drive not to mount?  I am thinking of getting 3 4TB drives to replace the array, thus reducing the array size.

 

Would you suggest that I;

1. replace one of the failed drive and see if the parity drive is mounted again, if it does, run the parity check and make sure the parity is up to date

2. replace the 2nd failed drive, again, re-run the parity check and all

3. replace the parity drive and again rebuild the parity

4. once the above is all done, remove the 1tb drive and rebuild the data from parity and reducing the array size

 

If replacing the failed drive doesn't mount the parity drive...what are my course of actions?

 

Thanks

[trurl]

At least some of your suggestions are not good, so don't do anything without further advice!!!

 

Go to Tools-diagnostics and attach the complete Diagnostics ZIP file to your NEXT post in this thread. 

[trurl]

Since you haven't replied with your diagnostics, I am going to give you some more information.

 

A disabled disk and an unmounted (or unmountable) disk, are not at all the same thing. A disabled data disk is usually emulated from the parity calculation, and the emulated disk is usually mounted even though the disabled disk is no longer being used.

 

But, parity is never mounted, whether or not it is disabled, because parity has no filesystem.

 

Your ideas:

1. Replacing one of the other disks won't fix disabled parity, and you can't even rebuild a replacement anyway because parity is disabled.
2. Basically the same as 1.
3. Closest to the right idea, but not clear there is anything wrong with the existing disk. Likely the solution is to rebuild parity, either to the same or a new disk, but can't say for sure without more information (your diagnostics)
4. After you get parity enabled again you can consider replacing other drives. We will have to wait and see.

PLEASE don't do anything without further advice.

[anhlon]

Good thing I checked with the experts here

Attached the the diagnostics zip file.  I was incorrect, there is one drive error from the dashboard.  So that's good.

nas-diagnostics-20200911-1150.zip

[trurl]

Parity and disk1 both need to be replaced. Probably they didn't both go bad at the same time. Have you been ignoring these problems? Do you have Notifications setup to alert you by email or other agent as soon as a problem is detected? Don't let one problem become multiple problems and data loss.

 

Do you know that in order to reliably rebuild every bit of a disk, whether parity or data, Unraid must be able to reliably read every bit of all other disks? Parity by itself cannot recover anything. And of course, currently parity is invalid anyway.

 

Do you have another copy of anything important and irreplaceable? This should probably be the first priority before attempting to rebuild anything since you have 2 disks that may make rebuilding difficult.

[anhlon]

this is due to me being lazy, setting it up a few years back and neglecting it.  aside from photos (~200gb) of it, the rest are replaceable.  let this be a lesson to me  

in the meantime i'll let the parity rebuild, get some new drives (don't replace it yet until i know how to proceed further).  setup notification and a sync of the data locally to an gcs or s3 bucket for a remote copy.

[trurl]

1 minute ago, anhlon said:

let the parity rebuild

It's already having problems reading disk1 for the parity rebuild.

 

Some other things I noticed, appdata, domains, system shares have files on the array. In fact, system is all on the bad disk1 which is probably why docker.img isn't mounting.

 

You want these shares all on cache and stay on cache so they won't have performance impacted by slower parity, and so they won't keep array disks spinning since these files are always open.

 

We can discuss what to do about that after your array is stable again. In fact, you might want to disable dockers and VMs and quit using your server at all until these disk issues are resolved.

[anhlon]

ok, both docker and vm are disabled now.

in the meantime, i'll go grab replacement drives at a local store.  may as well as upgrade the cache drive to a 1tb ssd.

[trurl]

10 minutes ago, anhlon said:

may as well as upgrade the cache drive to a 1tb ssd.

I would leave that until the array is stable.

[anhlon]

correct, that's the least of my concern at the moment, which i'll do very last.  i take that i should;

1. get the parity rebuild itself

2. replace disk1 (as you pointed out, it contains appdata, domains, system only -- which i could careless)

3. replace parity drive

[trurl]

29 minutes ago, anhlon said:

replace disk1 (as you pointed out, it contains appdata, domains, system only -- which i could careless)

I never said "only". Disk1 contains some files for each of your shares. You can see how much of each disk each share uses by going to User Shares and clicking Compute...

 

Go ahead and let parity rebuild complete on the original disk if it will. We will evaluate the next step as we see how things progress.