I have 2 hard drives from my very original RAID5 array from 2007, of the original 5 drives 3 have died and two are still working in the server.

 

I have a 2TB drive pre-cleared sitting there as a hot-swap ready for a dead drive, but I've started thinking i should be a bit proactive and just move the data off now before they actually die.

 

My plan would be to add the 2TB as disk8 and rsync the drive contents from the two disks to the 2TB drive.

 

I then need to figure out how to remove the two 500GB drives with as little downtime as possible.

 

The last time i ventured around with initconfig i got way in over my head and almost lost my array.

 

My only question would be, what is my safest way to do this before i start? I'm trying to find a post i used months and months ago about removing a drive without losing parity(i think).

No offense intended, but if you got in over your head with initconfig, you really don't need to be playing around with the convoluted process of removing drives without invalidating parity. Your best bet is to preclear your new drive for several passes, add the drive to the array, copy the data over, verify the copy, remove the old drives and set a new config to calculate parity with the new set of drives.

Maybe in over my head wasn't the right term, but last 2 times i used it, it didn't do what everyone told me it should do and stuff started happening in a bad way.

 

New hdd added as another drive, will copy over and then figure out what to do from there.

The simple solution is to unassign the old drive and assign the new in its place. Once the array is started the contents of the old drive will be reconstructed on the new.

 

The simple solution is to unassign the old drive and assign the new in its place. Once the array is started the contents of the old drive will be reconstructed on the new.

 

exactly. replace one drive, let it rebuild from parity, replace the second drive, let it rebuild.

 

While this is happening, i believe, the array will still function just slower as it is rebuilding the parity, so the only downtime is when the server is off while you replace the HDDs.

The simple solution is to unassign the old drive and assign the new in its place. Once the array is started the contents of the old drive will be reconstructed on the new.

 

exactly. replace one drive, let it rebuild from parity, replace the second drive, let it rebuild.

 

While this is happening, i believe, the array will still function just slower as it is rebuilding the parity, so the only downtime is when the server is off while you replace the HDDs.

 

Make a copy of the config folder before replacing a drive. This will help incase of a drive failure during the rebuild.

The simple solution is to unassign the old drive and assign the new in its place. Once the array is started the contents of the old drive will be reconstructed on the new.

That is the simplest, yes, but not the safest. If his second drive happens to fail while rebuilding, he is in trouble, and must do a trust parity procedure to try to rebuild the second drive with the first drive back in place. If his first drive fails while trying to recover from that failure, he is totally hosed.

 

By adding the drive and copying the data, he is reducing the stress on the old drives by only having to read the data portion, not a full disk read that is required to do a rebuild. After the data is copied from both drives he can do whatever he wishes with the old drives, safest thing being to remove them from the array and rebuild parity without them. If he wants to salvage them, several preclear cycles should either kill or confirm they are ok for the time being, and then they could be used wherever needed.

The simple solution is to unassign the old drive and assign the new in its place. Once the array is started the contents of the old drive will be reconstructed on the new.

That is the simplest, yes, but not the safest. If his second drive happens to fail while rebuilding, he is in trouble, and must do a trust parity procedure to try to rebuild the second drive with the first drive back in place. If his first drive fails while trying to recover from that failure, he is totally hosed.

 

By adding the drive and copying the data, he is reducing the stress on the old drives by only having to read the data portion, not a full disk read that is required to do a rebuild. After the data is copied from both drives he can do whatever he wishes with the old drives, safest thing being to remove them from the array and rebuild parity without them. If he wants to salvage them, several preclear cycles should either kill or confirm they are ok for the time being, and then they could be used wherever needed.

 

What happens if there is a data drive failure while rebuilding parity? There is no recourse.

 

As long as the config folder is backed up then a straight replace and rebuild is safer. If a drive fails during the rebuild than the original config and drive can be put back in place and then failure can be handled.

 

EDIT: The same technique could be used on the copy and rebuild parity method. There are far more steps and opportunity for user error with no additional safety. If copying is used then both old drives are read once. With rebuild only one of the old drives is read once. As long as the config folder is backed up then there is never a need for the un-reliable "trust parity" procedure.

What happens if there is a data drive failure while rebuilding parity? There is no recourse.

In my scenario, the only parity rebuild is after both old drives are removed after a successful copy, and since the 2TB drive has theoretically passed preclear, that chance is pretty remote, and the 2 old drives are still available for other recovery methods. My whole point is in trying to avoid using either of the 2 old drives to do a rebuild, as that is when the drive is stressed the most.

Data disks: 1, 2, 3

 

(Method 1) Copy and rebuild parity scenario:

 

Disks 1 and 2 are new copies. Start a parity build after removing the original drives. Now, disk 3 fails. The contents of disk 3 are unrecoverable at this point. Parity is not in sync and there is no backup of disk 3.

 

 

(Method 2) Replace and rebuild data disk scenario:

 

Backup config directory. Replace disk 1 and start a rebuild. Now disk 2 or disk 3 fails. Replace the original disk 1 and the original config directory. Replace and rebuild the failed drive.

 

Using method 2 the ability to recover from a single drive failure is maintained throughout the procedure.

Data disks: 1, 2, 3

 

(Method 1) Copy and rebuild parity scenario:

 

Disks 1 and 2 are new copies. Start a parity build after removing the original drives. Now, disk 3 fails. The contents of disk 3 are unrecoverable at this point. Parity is not in sync and there is no backup of disk 3.

 

 

(Method 2) Replace and rebuild data disk scenario:

 

Backup config directory. Replace disk 1 and start a rebuild. Now disk 2 or disk 3 fails. Replace the original disk 1 and the original config directory. Replace and rebuild the failed drive.

 

Using method 2 the ability to recover from a single drive failure is maintained throughout the procedure.

What do you think about this?

 

disable all plugins and make all shares read only - no writing to array allowed.

Backup flash drive.

Remove parity drive from array and init array so that it is operational but without parity.

add disk 4.

copy from disk 2 to disk 4 with MC disk to disk.

remove disk 2 and init array again but without disk 2 it is now operational with new disk.

add parity back to array and let parity rebuild.

re-enable all shares and plugins.

 

At any step except the rebuild at the end you can restore your array with the old disk 2 and removed parity by just plugging them back in and restoring the flash drive and restarting. 

 

With a single disk I wouldn't say this is any better than the standard method.  With multiple drives I could see an advantage since you aren't rebuilding one at a time and running all of the drives continuously.

Data disks: 1, 2, 3

 

(Method 1) Copy and rebuild parity scenario:

 

Disks 1 and 2 are new copies. Start a parity build after removing the original drives. Now, disk 3 fails. The contents of disk 3 are unrecoverable at this point. Parity is not in sync and there is no backup of disk 3.

Assuming an error free copy the first time from the original 2 data disks, it's not unreasonable to call the original 2 drives backup for the contents of drive 3. If the copy didn't work, the failed original drive could be replaced and rebuilt at that point, before the drives are removed and parity invalidated.

 

I still maintain that copying from the data area only is lower risk than performing a rebuild on one of the data drives relying on the other dodgy drive for a perfect total read. Also, I would much rather have 2 full copies of my data to attempt a recovery than rely on only one complete copy at a time.

Data disks: 1, 2, 3

 

(Method 1) Copy and rebuild parity scenario:

 

Disks 1 and 2 are new copies. Start a parity build after removing the original drives. Now, disk 3 fails. The contents of disk 3 are unrecoverable at this point. Parity is not in sync and there is no backup of disk 3.

Assuming an error free copy the first time from the original 2 data disks, it's not unreasonable to call the original 2 drives backup for the contents of drive 3. If the copy didn't work, the failed original drive could be replaced and rebuilt at that point, before the drives are removed and parity invalidated.

 

I'm making the same assumption that disks 1 and 2 are copied successfully. They are are not a backup for disk 3 without the the correct parity information. A failure of disk 3-20 during parity rebuild is not recoverable because parity is not in sync.

Data disks: 1, 2, 3

 

(Method 1) Copy and rebuild parity scenario:

 

Disks 1 and 2 are new copies. Start a parity build after removing the original drives. Now, disk 3 fails. The contents of disk 3 are unrecoverable at this point. Parity is not in sync and there is no backup of disk 3.

 

 

(Method 2) Replace and rebuild data disk scenario:

 

Backup config directory. Replace disk 1 and start a rebuild. Now disk 2 or disk 3 fails. Replace the original disk 1 and the original config directory. Replace and rebuild the failed drive.

 

Using method 2 the ability to recover from a single drive failure is maintained throughout the procedure.

What do you think about this?

 

disable all plugins and make all shares read only - no writing to array allowed.

Backup flash drive.

Remove parity drive from array and init array so that it is operational but without parity.

add disk 4.

copy from disk 2 to disk 4 with MC disk to disk.

remove disk 2 and init array again but without disk 2 it is now operational with new disk.

add parity back to array and let parity rebuild.

re-enable all shares and plugins.

 

At any step except the rebuild at the end you can restore your array with the old disk 2 and removed parity by just plugging them back in and restoring the flash drive and restarting. 

 

With a single disk I wouldn't say this is any better than the standard method.  With multiple drives I could see an advantage since you aren't rebuilding one at a time and running all of the drives continuously.

 

What happens if disk 3 fails during rebuild? A failure during any parity rebuild is not recoverable unless there is a copy of the failed disk.

What happens if disk 3 fails during rebuild? A failure during any parity rebuild is not recoverable unless there is a copy of the failed disk.

True for single drive replacement.  Once again I don't think my method is better for that.  However if you are replacing multiple drives you will have ALL drives running continuously rebuilding drive after drive.  That has got to increase the chance of a multi drive failure and the standard method cannot recover from that.  With mine method you have the potential of all drives spun down except for the drives involved in the transfer.  With the standard rebuild all drives spin to rebuild the data.  Then continue spinning when you go to drive 2, 3, 4, etc.  To me that would seem to be less ware and tear on the mechanical parts of the drive.

The odds of a single disk failure are much much higher than a double disk failure. unRAID is designed to recover from a single disk failure. Using the standard disk rebuild procedure (after backing up the config folder) maintains protection against a single disk failure throughout the replacement.

I'm making the same assumption that disks 1 and 2 are copied successfully. They are are not a backup for disk 3 without the the correct parity information. A failure of disk 3-20 during parity rebuild is not recoverable because parity is not in sync.

Disk 3 contains ONLY the data freshly copied from disk 1 and 2. How is a full verified copy of all the data on the removed drives NOT a backup?

This thread emphasizes one simple point about RAID arrays ==> they are NOT backups.    They simply provide fault tolerance that makes it much less of a hassle when a drive fails.    You should still have a backup of all of your data.

 

IF you have the data backed up, the simplest way to replace the 2 old (and smaller than you'd like) drives is to simply replace them one-at-a-time with a new drive and let UnRAID rebuild the drive.    This is a very simple process -- shut down;  replace the old drive with a new one;  and reboot.    The rebuild does take a fairly long time ... and yes, it has to read all the drives -- but it's a very reliable process.    The only issue is (as already noted) if you have a failure during the rebuild.

 

The alternative of adding the new drive to the array changes WHEN you are running "at risk" -- but doesn't eliminate the fact that you will still be doing so.    If you add the new drive, there's no "at risk" time during the preclear or while copying data from the old drives;  but when you then remove the old drives and re-init the array you're at-risk while parity is being computed.    Granted this is only one full read cycle of "at-risk" vs. 2 cycles if you replace both of the old drives.

 

The safest approach is to be sure all of your data is backed up BEFORE you do anything that involves running the array "at-risk".    I do my backups to a bunch of 2TB bare drives via an eSATA dock, and store them in drive boxes [These:  http://www.amazon.com/WiebeTech-DriveBox-Anti-Static-inch-Hard/dp/B004UALLPE/ref=sr_1_1?ie=UTF8&qid=1348909976&sr=8-1&keywords=wiebetech+drivebox ].  It does take extra time -- everything I write to the array I also write to my current backup drive ... and when it gets full I store it in a DriveBox and insert another drive in the dock.    But I have a complete backup of my 28TB UnRAID on drives that are safely tucked away in a fireproof & waterproof safe.    If I had a 2 (or more) drive failures, I'd simply have to replace the drives, then copy all of the backup drives to the newly initialized array.  (Yes, it'd take a long time, but only a few minutes of "my time" -- the rest would just be waiting for the copies to complete -- and there'd be NO data loss).

 

 

 

I'm making the same assumption that disks 1 and 2 are copied successfully. They are are not a backup for disk 3 without the the correct parity information. A failure of disk 3-20 during parity rebuild is not recoverable because parity is not in sync.

Disk 3 contains ONLY the data freshly copied from disk 1 and 2. How is a full verified copy of all the data on the removed drives NOT a backup?

 

This is not what I meant. Disk 1 and 2 represent the new disks that are copies of the disks to be replaced. Disk 1 and 2 are the freshly copied disks. Disk 3 represents all of the other disks in the array. A failure of any data disk other than disk 1 or disk 2 during the parity rebuild will result in data loss. (This is referred to as method 1) Let the chance of a single disk failure be f₁.

 

Rebuilding the disk 1 and then rebuilding disk 2 maintains correct parity throughout the replacement process. If any disk fails during the procedure, recovery is possible. (This is method 2) A 2 disk failure will result in data loss. Let the chance of a 2 disk failure be f₂.

 

Method one results in data loss if a single disk fails during the rebuild process. Method 2 allows recovery if a disk fails at any time during the procedure. The chances of 2 disks failing is much much smaller than the chance of a single disk failing. Method 2 maintains the failure recovery model of unRAID. Method 2 is safer than method one.

 

Method 1 chance of failure = f₁ + f₂ {The sum of the chance a 1 disk failure and the chance of a 2 disk failure.}

 

Method 2 chance of failure = 2 * f₂

{A 1 disk failure is recoverable so it's not included in the sum. The chance of a 2 disk failure is double because the procedure requires all disks be read twice. Once for each rebuild.}

 

2 * f₂ < f₁ + f₂