advice needed on massive array shrink

[CaptainTivo]

Hello,

I am planning to remove a number (probably 6) disks from my Unraid server array, retaining the data on the disks and putting them in "cold" storage backup.

The disks are all older (2014-2015) lower capacity (2,3 and 4 TB) disks.  The data on my array is of three types:

1) largely static (DVD and Bluray rips)

2) dynamic ( my home videos, photos, etc.)

3) backup data (that is duplicates of some of the other data on the array (dynamic)

 

My plan is to copy all the data from the disks to be removed onto newer 8, 10 and 12 TB disks.  In other words, I'm not reducing the capacity of the array, in fact I am increasing it.  The backup data is on dedicated shares on two disks that will not be removed.

 

The current state configuration is:

Backup disks: 1x 8TB and 1x 3 TB = 11 TB with 2.6 TBs free

Data disks: 2x 2TB, 3x 3TB, 2x 4TB, 2x 8TB + 10TB parity + 500GB SSD cache.  Total= 37 TB with 17.12 TB free.

 

In the discussion below, I call the disks to be removed "target" disks and the disks which will remain in the array "remainer" disks

The target disks are 2x 2TB, 3x 3TB and 1x 4TB, total of 17 TB.  They will be removed and place in Orico boxes for storage.

When all finished, the array will have 1x 4TB, 2x 8 TB, +10 TB parity.

 

My plan is this:

1) for each target disk:

    a) copy all data from a target disk to a remainer disk (e.g. rsysnc -avh  /mnt/diskx /mnt/disky)

    b) delete all data from the target that is not in the "static" category ( e.g.  "rm -rf" /mnt/disk/Videos/Home\ Video)

after doing all target disks, there should be enough remaining free space on the target disks to contain the static data residing in the remainer disks

2) copy all the static data on the remainer disks to the free space on the target disks using the "filelist" form of rsync (e.g. rysync -avhr --files-from disk9_filelist.txt /mnt/disk9 /mnt/disk1 ) where disk9_filelist.txt contains a list of directories under /mnt/disk9 that need to be archived on disk1 (target disk))

3) at this point, the target disks contain all the static data and none of the dynamic data. The remainer disks have all the data that was on all the target disks

Now remove the target disks and create a new config and rebuild parity.

 

Does this sound good?  Any changes you would make?  One possiblity is to copy and remove the disks one at a time and rebuild parity between each but I think that would put a lot of strain on the parity disk, right?

 

Thanks for any help.

 

 

 

[trurl]

13 minutes ago, CaptainTivo said:

Does this sound good? 

Yes, only one new config / parity rebuild at the end.

[CaptainTivo]

Ooops.  I was wrong: I don't have enough free space left on the array to copy all the data from the target disks onto the remainer disks.

And I currently have no free SATA ports so that I will have to remove at least one disk to add a new empty disk.

I have a pre-cleared 8 TB.  Here is the procedure I plan now.

 

 1) copy all data from the target disk to a remainer disk

2) delete the "dynamic" data from the target

3) copy some files from the remainder disks that I want to archive to fill the free space on the target disk

4) do the "Remove Drives then Rebuild Parity" method found here: https://wiki.unraid.net/Shrink_array

this assumes that you don't want to keep the data on the disk to be removed, and this is true since I have already copied the data from this disk to one of the remainer disks, right?

5) add the new 8 TB pre-cleared disk using this procedure: https://wiki.unraid.net/UnRAID_6/Storage_Management#Adding_disks

This way, it will not rebuild parity yet again because the drive is pre-cleared.

6) now proceed with the procedure outlined in my previous post since there will now be enough free space on the remainer drives to hold all the data currently on the target disks. (requires yet another parity rebuild)

 

This means that I will have to rebuild parity 2 times instead of one but I don't see how I can avoid that.

 

I believe that I can remove more than one target disk (after copying the data, of course) before doing the Shrink Array procedure.  This would give me more empty SATA ports and allow me to do a pre-clear on yet another disk at the same time the parity rebuild is going on.  Is this a good idea?

 

Sorry for being so verbose, but I find that with Unraid, details really matter!

 

Thanks.

 

 

 

[Jaster]

I just did a similar procedure and it went without issues;

First I installed, preclean and added larger disks.

Now I excluded all target disks from all shares.

I used unBalance to move all the data from the target disks to the remainers.

Now I did a new config but removed the target disks and started a parity check. Remember to remov all the excludes from all the shares.

Now I just used the target disks as unassogned disks and reformated them to another filesystem. You could also preclean them to make sure they are still working well. Copy the data once pariry is finished. I have about 30tb of data and the whole procedure took about 120hrs.

[trurl]

1 hour ago, Jaster said:

new config but removed the target disks and started a parity check sync

FTFY