Migration done right

[xtrips]

Hello,

 

I have an Unraid machine running with 9 HDDs including parity and one SSD drive for cache.
5 active dockers and a few plugins.

In a few days I will be ready to migrate to a completely new machine assembled from scratch.
Here are the steps I wish to perform:
1- Move everything to the new machine "as is" and make it work
2- Migrate the Cache role and content from my 480Gb SSD to a new 1Tb NVMe
3- Start to add other HDDs taken from another aging NAS, finally reaching a total of 18 HDDs in the new enclosure

What am I in for? What surprises await for me? Please give it all to me. Warnings, advices, recommendations.
The most elementary questions come to mind. Are there any special considerations with a much newer generation motherboard? Settings? Parameters? Maybe bios? Maybe some testing of the new machine before I start to move everything in and make my Unraid temporarily  unavailable?

I cannot install all the new HDDs at once. I have to do that gradually as I also move the data in. How should I proceed and do that right?
I admit I am a bit concerned.

Thank you 

[itimpi]

#1 should "just work" if you have no VMs with hardware being passed through to them and n/ne of the disk controllers are RAID ones.

 

#2 should be easy enough - but others may give some advice on best way to achieve it.

 

#3 Not sure if these drives have data on them but if so basically this is just a case of copying the files to the array.   However you need to provide more information on how they might be connected and what their current format is as this will affect any advice.  Have you run any sort of confidence check on the state of these old drives?

[xtrips]

#1 should "just work" if you have no VMs with hardware being passed through to them and n/ne of the disk controllers are RAID ones.
 
#2 should be easy enough - but others may give some advice on best way to achieve it.
 
#3 Not sure if these drives have data on them but if so basically this is just a case of copying the files to the array.   However you need to provide more information on how they might be connected and what their current format is as this will affect any advice.  Have you run any sort of confidence check on the state of these old drives?

About #3, the HDDs being transferred to Unraid after their data has been copied via the network are destined to be formatted by Unraid and added to the array, same as if they were new virgin drives.


Sent from my iPhone using Tapatalk Pro

[itimpi]

42 minutes ago, xtrips said:

 

About #3, the HDDs being transferred to Unraid after their data has been copied via the network are destined to be formatted by Unraid and added to the array, same as if they were new virgin drives.


Sent from my iPhone using Tapatalk Pro

 

OK 

 

That leaves the question about whether you are sure they are ‘good’ drives as adding unreliable drives to an unRaid array is never a good idea.  If you are not sure then after copying the data off you could either run something like ‘preclear’ or the manufacturer’s test software against them to check them out before adding them.  If you are happy with their status just add them and let unRaid run the automatic ‘clear’ operation which is also a minimal level of confidence test.

[xtrips]

5 hours ago, itimpi said:

OK 

 

That leaves the question about whether you are sure they are ‘good’ drives as adding unreliable drives to an unRaid array is never a good idea.  If you are not sure then after copying the data off you could either run something like ‘preclear’ or the manufacturer’s test software against them to check them out before adding them.  If you are happy with their status just add them and let unRaid run the automatic ‘clear’ operation which is also a minimal level of confidence test.

Of course I will check the disks first. Thank you.
I expected more technical advice, about bios settings, how to make sure I take advantage of the speed. Before I was limited to 3 Gbps SATA. Now I will have 6 Gbps, and using an NVMe cache disk should yield better response. 
I still need help about migrating my old cache to the new cache. My first guess would be to shut down the Dockers before replicating all the cache content to the NVMe. Will that be enough after I designate it as the new cache?

Edited February 27, 2021 by xtrips

[trurl]

14 minutes ago, xtrips said:

old cache to the new cache

https://wiki.unraid.net/index.php/Replace_A_Cache_Drive

[trurl]

9 hours ago, xtrips said:

Start to add other HDDs taken from another aging NAS, finally reaching a total of 18 HDDs in the new enclosure

Don't install disks just because you happen to have them. Only add what you need for capacity. Each additional disk is an additional point of failure, requires more hardware to support and more power. Older smaller disks often aren't worth bothering with, fewer and newer larger disks perform better and don't require as much hardware and power. If you plan to have many disks be sure to setup dual parity.

[xtrips]

Getting no red light warnings at all I suppose putting all the HDDs, the SSD, the NVMe, the PCI SATA ports extenders, all in place and booting from the Unraid thumbdrive should be pretty straight forward, right?

[xtrips]

7 minutes ago, trurl said:

Don't install disks just because you happen to have them. Only add what you need for capacity. Each additional disk is an additional point of failure, requires more hardware to support and more power. Older smaller disks often aren't worth bothering with, fewer and newer larger disks perform better and don't require as much hardware and power. If you plan to have many disks be sure to setup dual parity.

At last something worth noting! What is dual parity? Is there a manual to set that up?
Most of the HDDs will be used as in fact I am merging 2 big NAS machines (Unraid and Readynas) into one.

[theruck]

also do not expect the nvme performing as nvme with unraid. you will be glad if you will be able to copy 100 MB/s to your machine so you do not need 1500 MB/s unles you plan to use a lot of VMs utilizing the high iops of the nvme drive

[xtrips]

16 minutes ago, theruck said:

also do not expect the nvme performing as nvme with unraid. you will be glad if you will be able to copy 100 MB/s to your machine so you do not need 1500 MB/s unles you plan to use a lot of VMs utilizing the high iops of the nvme drive

That is bad news. I do not use VMs at all. Based on this fact my only incentive to upgrade my cache drive to an NVMe will be the size and the fact that it will free one more slot for a future HDD? No speed gain at all?

[trurl]

46 minutes ago, xtrips said:

What is dual parity?

There is a parity2 slot you can assign 2nd parity disk to. Each parity disk must be at least as large as the largest single data disk.

[xtrips]

Just now, trurl said:

There is a parity2 slot you can assign 2nd parity disk to. Each parity disk must be at least as large as the largest single data disk.

I understand. This is useful only if 2 HDDs fail at the same time? Pretty rare I suppose. Or are there any more advantages?

 

[theruck]

6 minutes ago, xtrips said:

That is bad news. I do not use VMs at all. Based on this fact my only incentive to upgrade my cache drive to an NVMe will be the size and the fact that it will free one more slot for a future HDD? No speed gain at all?

where would you expect the speed gain?

[xtrips]

2 minutes ago, theruck said:

where would you expect the speed gain?

Dunno. Copying files to Unraid from the network? Or multiple downloads to Unraid at the same time? Didn't really think about it.
So if there is no speed gain but I still gain by saving a SATA slot, maybe I could use an SSD M.2 instead if they are cheaper.......

[trurl]

There are several things to consider with cache and speed. Cache (or additional pools with 6.9), your Dockers and VMs are usually configured to use that space for performance, and so array disks can spin down. Cache is also used to cache user share writes to be moved to the slower array later.

5 minutes ago, xtrips said:

I understand. This is useful only if 2 HDDs fail at the same time? Pretty rare I suppose. Or are there any more advantages?

Often people will disturb connections when replacing a disk and dual parity can sometimes help with that. And, sometimes another disk will actually begin to have problems while rebuilding another disk. Be sure to setup Notifications to alert you immediately by email or other agent as soon as a problem is detected. Unraid monitors SMART for all disks (unless you try to use RAID controller, don't).

 

[theruck]

1 Gbit network = max 120 MB/s - you can achieve that with a 10year old SSD drive

+there is large software overhead with unraid anyway

Edited February 27, 2021 by theruck

[xtrips]

1 minute ago, theruck said:

1 Gbit network = max 120 MB/s - you can achieve that with a 10year old SSD drive

 

What was I thinking.....

[trurl]

Just now, theruck said:

1 Gbit network = max 120 MB/s - you can achieve that with a 10year old SSD drive

But you probably can't achieve that writing directly to the parity array, so you configure it to write to cache instead.

[theruck]

you can use the speed and iops of nvme mostly at VMs if you have a database server etc. but for storing files or as a cache drive it is just overkill