My first attempt at a big move to new hardware, and encrypting drives.

[alexdodd]

Ok, so i'm wondering how I best approach this.  This is my first big move of data and new hardware, so a little sense check is needed.

Background: I have an exisitng old build in a ramshackle over extended dell T20.  Although it has been going strong for 5+ years apparently with minimal intervention, and 2 house moves! It is groaning and creaking under the way its treated and is passed its use-by.  It needs to be retired to something supportive and less prime time.  Backups perhaps.

Main Goal:  Move all drives to new (to me) supermicro 3u with new hardware.
Sub Goal 1: Encrypt drives
Sub Goal 2: Add an ssd drive I have recently free'd up for cache
Sub Goal 3: Restructure my folders for heavy disk use applications and long term storage.  Currently all shares, appdata and all dockers share all the parity protected drives, this probably isnt right, I think I should have my frequently changing data outside of parity operations (downloads etc?) probably on a cache drive i'm about to install above^.

Array currently looks like this:
  

I have spare unassigned drives as hotspares to lean on, if necessary:

 

Note: These are seagate SAS drives on an LSI card in IT mode, and sadly that means they don't appear on unassigned devices plugin, which does potentially make my life a little harder.

My thinking is like this:

1. Changed default FS to BTRFS Encrypted
2. I added Disk 7 and formated (BTRFS Encrypted)
3. Move all on Disk 2 over to DIsk 7 with unbalance*
4. Wipe Disk 2 partition table (--How?)
5. Format Disk 2 with BTRFS Encrypted through unraid
6. Move all on Disks 5, 6 to the Disk 2 (now BTRFS Encrpyted) with unbalance
7. Wipe Disks 5, 6 partition tables
8. Format Disks 5, 6 with BTRFS Encrypted through unraid
9. Move all on Disks 1, 3, 4 to the Disk 5 with unbalance
10. Wipe Disks 1, 3, 4  partition tables
11. Remove Disks 1, 3, 4 from the array, shrink it and rebuild parity.
12. I need to then highlight which drives are which in the tower (how/best practice for now and then future?)
13. Drives Parity, Disk 2, 5, 6, 7 need to be transplanted to the new case and motherboard.
14. USB drive moves over (it likely doesn't like the new hardware)
15. I have to apply for a new key or similar.
16. Look at where my appdata and dockers are pointed and set to download etc...

 
Questions:

1. I think there are various stages between 1-11 i need to make a "new config" but i'm not exactly sure,  so i need a little guidance on that.
2. Is there a way I can automate the moving of 3 disks insteaf of invoking one at a time with unbalance?
3. How do i format the drives without using unassigned devices to remove the partition.

 

*I had a false start with the first simple stage of the operation above (see: here) so I thought before i lose everything or totally mess something up its best to sense check my madness.

Many thanks for all help and input!

Edited April 18, 2020 by alexdodd

[JorgeB]

16 hours ago, alexdodd said:

I think there are various stages between 1-11 i need to make a "new config" but i'm not exactly sure,  so i need a little guidance on that.

New config is needed at step 11, also no need to wipe previous disks/partitions, just setting them to an encrypted filesystem is enough, Unraid will offer to format it at next array start.

 

16 hours ago, alexdodd said:

Is there a way I can automate the moving of 3 disks insteaf of invoking one at a time with unbalance?

If it's a single share on those disks you could rename it on all 3 disks and move it together, but it not be as simple as using Unbalance.

 

16 hours ago, alexdodd said:

How do i format the drives without using unassigned devices to remove the partition.

See above.

[alexdodd]

Thanks! 

Slight change of plan, I'm reconsidering my btrfs encrypted choice now 😬. I have no reason to believe i'd use its extra features and I don't have a UPS. Its abundantly clear I have no clue what i'm doing under the hood.  Sounds like a better filesystem due to checksums and snapshot mind.

So my Disk 7 is currently formatted to btrfs encrypted and now full, if i were to change my default in unraid to XFS encrypted now, can i have XFS encrypted drives in the pool at the same time, whilst I move from BTRFS to XFS encrypted?

I would presume as unraid doesn't really care about filesystems right?

[itimpi]

4 minutes ago, alexdodd said:

would presume as unraid doesn't really care about filesystems right?

Correct.   You can have any mix of the supported file systems in the array and Unraid will function just fine.

[alexdodd]

Ok, i'll carry on what i'm doing, but for my own sake its probably better I use XFS encrypted, and finish with al drives like that

Cheers!

[alexdodd]

Amazing. Dodgy disk 4 has been marked failed presumably due to the disk thrashing highlighting its quality.

I want to just remove it from the equation since i have the free space, and shrink the array, if i put no device in its place, start and rebuild parity that is correct right?



 

Edited April 19, 2020 by alexdodd

[alexdodd]

Wait as usual i'm confused, and I dont want to destroy parity.
This is where i'm at:

Disk 5 is empty, because i've just moved everything off in the last unbalance move.  It's still in xfs, can i format it in xfs encrypted, and then move the emulated Disk 4 over with unbalance.  Then when emulated disk 4 is empty, create a new config carry them all forward, remove disk 4 and mark parity not valid.

I'm just not sure how i format Disk 5 to encrypted xfs in the above workflow?

 

\\Edit:Or do i have to move emulated Disk 4 to Disk 5 formatted in xfs as is?

Edited April 19, 2020 by alexdodd

[JorgeB]

18 hours ago, alexdodd said:

if i put no device in its place, start and rebuild parity that is correct right?

It will start with an emulated disk4 (assuming parity is valid), you don't want to rebuild parity now if there's still data there.

[alexdodd]

7 hours ago, johnnie.black said:

It will start with an emulated disk4 (assuming parity is valid), you don't want to rebuild parity now if there's still data there.

Yep, so i started with it emulated, moved it away to other drives, started a new config carried the config forward without the dead drive, rebuilt partiy.  I think I might actually be starting to get the hang of this without worrying so much.  I doubt myself everytime in case I screw it up 😂

So now I have an empty 2tb drive (sdg/Disk2), but formated in XFS, still added to the array.  However I want this in an encrypted FS.

In my head i need to create a new config, but if I assign the drive to the same slot, how do i force unraid to format it?

Is there something i'm missing?

Edited April 20, 2020 by alexdodd

[JonathanM]

1 minute ago, alexdodd said:

So now I have a 2tb drive with nothing on, formated in XFS, still added to the array.  However I want this in an encrypted FS.

In my head i need to create a new config, but if I assign the drive to the same slot, how do i force unraid to format it?

Is there something i'm missing?

Yep. No need to do a new config, just stop the array, and set the desired file system type on that drive to encrypted. When you start the array, the drive should show up as unmountable, and you will need to confirm the prompt to format it.

 

New config is only for recalculating parity, and since parity has no concept of files or file systems, it's not needed to redo parity when changing formats. Parity stays valid through the entire process.

[itimpi]

2 minutes ago, alexdodd said:

Yep, so i started with it emulated, moved it away to other drives, started a new config carried the config forward without the dead drive, rebuilt partiy.  I think I might actually be starting to get the hang of this without worrying so much.  I doubt myself everytime in case I screw it up 😂

So now I have a 2tb drive with nothing on, formated in XFS, still added to the array.  However I want this in an encrypted FS.

In my head i need to create a new config, but if I assign the drive to the same slot, how do i force unraid to format it?

Is there something i'm missing?

It is easier than that

• stop array
• click on drive and change format to the encrypted variant you want
• start array - disk will now show as unmountable
• use the option to format unmountable disks.

[alexdodd]

Yep got there with @jonathanm response.  You guys are all legends.  Thanks so much for the replies to my rather basic questions.  Really appreciate everything that happens in this little community, and the product unraid offer.

Its a testement to its reliability that i just set and forget it mostly untill i have to do things like this!