Replacing multiple smaller disks with a single larger one???

[wgstarks]

I recently purchased a Seagate 8TB archived drive for use as Parity (rebuilding parity now) and plan to use the previous parity drive (4TB WD Red) to replace 2-1TB data drives and a 2TB data drive.

 

I know I can just add the 4TB drive to the array and use unbalance or rsync to transfer all the data from the drives being removed. I'm wondering if it would save time to pull the 2TB drive, add the 4TB drive as a replacement, restore data from parity, and then transfer data from the two 1TB drives.

 

Would this be any faster? Not sure if it would save any time or if it would be just as easy to just add the 4TB drive and transfer data from each old drive and just do a new config after each drive is emptied?

 

Opinions?

[John_M]

It probably would be faster than rsync but before starting a rebuild I would want to make sure the parity I'd just built was good so I'd run a parity check first, which takes time, of course.

 

[wgstarks]

Got plenty of time really. No disk failures or anything like that.

[FreeMan]

Time is fairly cheap insurance to ensure there aren't any issues with the parity build!

 

Also, remember that a disk rebuild will rebuild the file system that was on the old disk. If you're migrating away from all reiserfs disks in this process, you'll need to add the old parity disk as a new member of the array, format it with your fs of choice, then copy data. If, of course, you're already on the new file system, then this is a moot point.

[John_M]

I have to say I'm confused by the direction this thread has taken. I thought the original question was specifically about saving time, not about changing file systems.

[FreeMan]

I have to say I'm confused by the direction this thread has taken. I thought the original question was specifically about saving time, not about changing file systems.

 

It is about speed, I just wanted to put the reminder there for OP and for any others who may stumble upon it in the future. I've seen a number of people confused by this (even in the sticiked thread on converting from resierfs).

[wgstarks]

Actually, I switched everything to XFS quite a while back. This is just replacing older smaller drives with larger faster drives. I'm hoping that my parity check times will improve with all drives being 4TB+.

[John_M]

I meant this:

 

Got plenty of time really.

[wgstarks]

I meant this:

 

Got plenty of time really.

I guess it does sound contradictory. I'll be checking parity after the build is finished since that's the safest way IMO. Had planned on that from the start. I don't really want to waste a bunch of time with the disk replacement but don't want to cut any corners where data loss might result.

[truckerCLOCK]

I recently purchased a Seagate 8TB archived drive for use as Parity (rebuilding parity now) and plan to use the previous parity drive (4TB WD Red) to replace 2-1TB data drives and a 2TB data drive.

 

I know I can just add the 4TB drive to the array and use unbalance or rsync to transfer all the data from the drives being removed. I'm wondering if it would save time to pull the 2TB drive, add the 4TB drive as a replacement, restore data from parity, and then transfer data from the two 1TB drives.

 

Would this be any faster? Not sure if it would save any time or if it would be just as easy to just add the 4TB drive and transfer data from each old drive and just do a new config after each drive is emptied?

 

Opinions?

 

8TB be prepaired for some waiting..LOL I just upgraded to a 6TB and parity rebuild took a little over 16 hrs

[wgstarks]

Yeah. Been running for 16.5 hours and showing 5 more to go.

[garycase]

r.e. the fastest time ...

 

Consider the following:

 

=>  You're doing a parity sync; and are then going to do a parity check to confirm all is well.    This time is "baked in" and doesn't have any impact on the time to do the file transfers.

 

=>  I presume your plan is to remove the 2 1TB drives and the 2TB drive after you've copied all of the data ... so this will eventually require a "New Config" and another round of parity sync and parity check [There ARE ways to avoid this and keep parity valid -- but it wouldn't be any quicker]

 

So ...

 

(1)  If you plan to do a New Config to remove the 3 drives, the fastest approach is probably to do the following:

 

  (a)  Do a New Config NOW (after the parity sync/check are done so you KNOW all of your other disks are good), with the old 4TB parity included in the array, but the 3 drives you want to copy date from NOT in the array.    This will eliminate the need to clear the old 4TB drive (it will just need formatted -- a couple minute process)  You'll need to do another parity sync/check cycle -- but this is eventually going to be needed anyway, so just get it done.

 

  (b)  Use the "Unattached Devices" plugin to copy the data from the 3 smaller drives to the array at your leisure.  [if you don't have enough ports for them all, just install one at a time]

 

(2)  If you are going to try and preserve parity, you'll need to do the following:

 

  (a)  Add the 4TB old parity drive to the array after you've completed the new parity sync/check cycle.    This will require that it be cleared (UnRAID will do this for you) and then formatted ... so it will take a fairly long time (many hours).

 

  (b)  Copy the data from the smaller drives to the 4TB drive ==> be CERTAIN that you avoid the "user share copy bug" so you don't lose any data (I'm reasonably sure you already know how to do this)

 

  ©  Write zeroes to all 3 of the smaller drives (this will take a long time, but is the key to preserving parity on your array when you remove them).    Be sure to do this in accordance with the thread that outlines how to do it and preserve parity.

 

  (d)  Do  a New Config that excludes the 3 smaller drives and check the "parity is already valid" box => which will be true since all of the data on the 3 drives has been zeroed.    I'd nevertheless do a parity check after starting this new configuration, just to be certain everything went well.

 

[garycase]

The process for clearing a drive while preserving parity is outlined here:  http://lime-technology.com/wiki/index.php?title=Shrink_array

 

See the section labeled:  The "Clear Drive Then Remove Drive" Method

[garycase]

Just read through the script Rob outlined in his article, and it says it's designed to only remove one drive at a time.

 

However, if you clear multiple drives, the New Config with "parity is already valid" will work just fine.  Note sure if Rob's script will allow that, but if you use the Linux command outlined in the section labeled "Alternate Procedure for Linux proficient users" you can clear all 3 drives before doing the New Config.    HOWEVER => if you do that be CERTAIN your drive references are correct -- otherwise you'll clear one of your other data drives !!!  (and since this is done while maintaining parity, that error would be completely unrecoverable ... reconstructing the drive would just result in all zeroes on the reconstructed drive)

 

 

[trurl]

wiki: Replacing Multiple Data Drives with a Single Larger Drive

 

 

 

[garycase]

The wiki article does indeed do it the fastest way, but it does it completely without parity protection for all of the copies (the first method) or with parity protection during the copies but then runs at risk while doing a new parity sync (2nd method).

 

There's not really much risk involved, however, as long as you do NOTHING else on the array while doing the copies.  The reason is simple:  As long as you still have the original, unmodified small disks AND the unassigned (and unmodified) parity drive, you could always do a New Config with the "parity is already valid" and restore the original setup, should one of your data drives fail during the copying process.    The only risk is at the end, when you assign a parity drive and have to wait for the parity sync/check cycle.    [And this could even be avoided if you have a 2nd 8TB drive to do the new parity sync to => since you'd still have the ability to restore the original configuration.  Not necessarily a bad idea to do that -- assuming everything went well, you could then assign your current 8TB parity drive as a 2nd parity drive and have a dual parity array going forward.]

 

[wgstarks]

wiki: Replacing Multiple Data Drives with a Single Larger Drive

Thanks trurl. Don't know why Google didn't find that. Think I'll use the "safer method" outlined there.

[landS]

While not difficult per the guide...  This would be a nice GUI feature :  choose X disks for depreciation ,  choose new single replacement disk.    Adds new disk to configure,  copy all data,  removes old from configuration, 

[wgstarks]

The wiki article does indeed do it the fastest way, but it does it completely without parity protection for all of the copies (the first method) or with parity protection during the copies but then runs at risk while doing a new parity sync (2nd method).

 

There's not really much risk involved, however, as long as you do NOTHING else on the array while doing the copies.  The reason is simple:  As long as you still have the original, unmodified small disks AND the unassigned (and unmodified) parity drive, you could always do a New Config with the "parity is already valid" and restore the original setup, should one of your data drives fail during the copying process.    The only risk is at the end, when you assign a parity drive and have to wait for the parity sync/check cycle.    [And this could even be avoided if you have a 2nd 8TB drive to do the new parity sync to => since you'd still have the ability to restore the original configuration.  Not necessarily a bad idea to do that -- assuming everything went well, you could then assign your current 8TB parity drive as a 2nd parity drive and have a dual parity array going forward.]

 

I agree. The "safer method" seems simpler and as you say, I'll still have the old drives if needed.

[garycase]

The real key to being "safe" is to always be sure you're working with a good array.  You're doing that by doing a parity check after your initial parity sync ... and by then doing the copies in a parity-protected manner.

 

Once the copies are all done, I would add one more step to the "safer" method indicated in the wiki article => Do yet another parity check BEFORE doing the New Config that removes the final two drives.    This will ensure that there were no "hiccups" during the copies, and that all of your disks are good at that point => certainly better to know about any issues then rather than while you're doing a new parity sync for the new config !!

 

[wgstarks]

The real key to being "safe" is to always be sure you're working with a good array.  You're doing that by doing a parity check after your initial parity sync ... and by then doing the copies in a parity-protected manner.

 

Once the copies are all done, I would add one more step to the "safer" method indicated in the wiki article => Do yet another parity check BEFORE doing the New Config that removes the final two drives.    This will ensure that there were no "hiccups" during the copies, and that all of your disks are good at that point => certainly better to know about any issues then rather than while you're doing a new parity sync for the new config !!

 

Sounds good to me. If I exclude the new 4TB drive from all shares, that will prevent any problems with duplicated files outside of unRaid server right? And then remove the exclusion when all checks are finished and I'm ready to do a new config to remove the old drives from the array. Don't want to generate a bunch of duplicates in my Kodi database.

[garycase]

If you exclude the new drive from the Global shares that should do the trick.

 

Probably the SAFEST way to do this within the array is to NOT use the same folder name as an existing share.  e.g. if you have a share named "Movies" on one of the drives you're copying data from; create a folder named MoviesX" on the new drive; and copy all of the data from the old drive's "Movies" folder to "MoviesX" on the new drive.  Ditto for all folders on the drives you're copying from.

 

Then, after you've done the New Config without the old drives, rename the "MoviesX" folder to "Movies" (and any other folders you've done this for) and re-enable that drive in Global shares.