Transitioning from nas4free to unraid in same NAS

[Ortoch]

I currently have a 36 drive NAS that is about 60% full and I want to figure out how to transition from the nas4free utility to unraid so I can take advantage of the dockers and eliminate my VMware management server. The NAS device has 2 8GB solid state drives in it and 36 4TB 7200 rpm drives that make up the pool. My desire now is to load unraid onto a thumb drive and somehow boot the unraid on this device and slowly fail out drives from the nas4free software as I transfer files over to the unraid pool that grows as I fail out each drive.

 

Is this plan feasible at all? nas4free sits on one of the internal 8GB flash drives of that matters but just hoping that someone can offer advice on how to transition this over. 
 

to work with I have a Dell R710 and a desktop PC that I can configure any way needed but I just need some advice on how to approach this.

 

thanks!

[Ortoch]

Anyone able to offer an idea?

[Hoopster]

On 10/20/2019 at 10:11 PM, Ortoch said:

I currently have a 36 drive NAS that is about 60% full and I want to figure out how to transition from the nas4free utility to unraid so I can take advantage of the dockers and eliminate my VMware management server. The NAS device has 2 8GB solid state drives in it and 36 4TB 7200 rpm drives that make up the pool. My desire now is to load unraid onto a thumb drive and somehow boot the unraid on this device and slowly fail out drives from the nas4free software as I transfer files over to the unraid pool that grows as I fail out each drive.

I am not familiar with how nas4free works, but, there are some things you should consider as you formulate your migration plan:

• An unRAID array requires at least one data drive in the array.  Parity and cache drives are optional and can be added before or after data transfer is complete
• When a disk is added to the unRAID array, it MUST be formatted.  You will lose all data on the disk, it is not possible to migrate data in-place without formatting
• Backup your nas4free data disk by disk if you wish and then you can remove a disk from nas4free, add it to the unRAID array and copy the data back to the disk.  Optionally you can do this a few disks at a time depending on how you want your data distributed on the array (see next point)
• unRAID user shares can span multiple disks, so, you have to decide if you want a share to correspond to just one disk or span multiple disks.  This may factor into your data migration strategy - make sure you understand how the Allocation Method and Split Level settings work in unRAID shares
• When migrating data to the array, it is often preferred to leave out the parity disk(s) and add parity once the bulk of the migration is complete.  This will speed up data transfer times and parity will be calculated once the parity disks are added
• When adding a cache drive to the array, it should be an SSD.  The cache drive has a couple of main purposes (you decide how to use it).  It will cache writes to the parity-protected array and/or it can be the storage location for docker container configuration data and/or VMs
• Migrate your data first and then worry about docker and/or VM configurations

My two cents. Other opinions may vary.

[JonathanM]

On 10/21/2019 at 12:11 AM, Ortoch said:

slowly fail out drives from the nas4free software as I transfer files over to the unraid pool that grows as I fail out each drive.

That seems like an incredibly risky procedure. What sort of backups do you have in place should you lose data trying to do this?

 

Perhaps it would be wiser to purchase a few new much larger drives and start the unraid array with new disks, 10TB seems to be the new sweet spot for pricing this season.

[Ortoch]

13 hours ago, Hoopster said:

I am not familiar with how nas4free works, but, there are some things you should consider as you formulate your migration plan:

• An unRAID array requires at least one data drive in the array.  Parity and cache drives are optional and can be added before or after data transfer is complete
• When a disk is added to the unRAID array, it MUST be formatted.  You will lose all data on the disk, it is not possible to migrate data in-place without formatting
• Backup your nas4free data disk by disk if you wish and then you can remove a disk from nas4free, add it to the unRAID array and copy the data back to the disk.  Optionally you can do this a few disks at a time depending on how you want your data distributed on the array (see next point)
• unRAID user shares can span multiple disks, so, you have to decide if you want a share to correspond to just one disk or span multiple disks.  This may factor into your data migration strategy - make sure you understand how the Allocation Method and Split Level settings work in unRAID shares
• When migrating data to the array, it is often preferred to leave out the parity disk(s) and add parity once the bulk of the migration is complete.  This will speed up data transfer times and parity will be calculated once the parity disks are added
• When adding a cache drive to the array, it should be an SSD.  The cache drive has a couple of main purposes (you decide how to use it).  It will cache writes to the parity-protected array and/or it can be the storage location for docker container configuration data and/or VMs
• Migrate your data first and then worry about docker and/or VM configurations

My two cents. Other opinions may vary.

Hoopster and johnathan thank you both for the replies. My only remianing question would be is there a way to boot a device with nas4free or any other nas management software and then be able to see unraid from a thumb drive simultaneously? Based on your responses that’s my only remaining unknown hurdle. I need to be able to see the current nas and unraid at the same time but only have one chassis that can see the drives. Since unraid can’t read ZFS formatting I need both nas managers active at the same time so I can fail out drives from the original nas and then add them to unraid and migrate data over.

 

thanks again!

[Ortoch]

13 hours ago, jonathanm said:

That seems like an incredibly risky procedure. What sort of backups do you have in place should you lose data trying to do this?

 

Perhaps it would be wiser to purchase a few new much larger drives and start the unraid array with new disks, 10TB seems to be the new sweet spot for pricing this season.

I have a dozen + empty 4TB drives so the high value data will be copied directly over before I start deleting copied data and failing drives out so it will be less valuable data that will be moved over in one drive chunks as I smartfail out till it’s all migrated. 

[JonathanM]

So do you have backups of all high value data stored elsewhere? Fault tolerance provided by a NAS is not a backup, plenty of ways to lose or corrupt data without a drive going bad.

[Ortoch]

47 minutes ago, jonathanm said:

So do you have backups of all high value data stored elsewhere? Fault tolerance provided by a NAS is not a backup, plenty of ways to lose or corrupt data without a drive going bad.

Its media that I have sotred elsewhere but its not fast enough to stream from the remote site and would take quite a while to rebuild so loosing the data locally would suck but it would not be the end of the world. So in short yes its backed up/recreatable in a pinch. The biggest Issue I've found with backing it all up properly is the size since I am dealing with over 100TB of data I cant find a reasonable storage service to back it up even with high compression that is practical due to up/down speeds and/or cost.

I'm down to the point where having unraid on the same chasis as my storage is key since I cant afford to keep running this R710 as well as the NAS so regardless of risks I have to start the migration this weekend I just need a way to get both nas4free and unraid running at the same time to start slinging data from one to the other.

[Ortoch]

I'm about to give up on this project not even sure its feasible at this point. 

If anyone else has any ideas how I can get 60TB off my current array and onto unraid on the same array let me know but I need to take a break been working on this for two days now and have gotten nowhere hah.

 

-Ort