Final migration from old server to new server(s) with HDD/SSD upgrade - proper process?

[resentedpoet]

Server 1 (new)

 

T430 currently with UNRAID installed and 2x 300GB SAS HDD's and 1x SSD (512gb) for Parity (testing config)

Currently running some small docker containers for Frigate, Jellyfin (testing), Wyze-bridge, and Dell iDrac controller

 

Server 2 (old)

i7-4770k with a mix of HDD's for media content and VM's with 1x 256GB SSD for CACHE/Docker containers.

Running Plex, updatekuma, komga, pihole, a few minecraft servers and my Home-Assistant VM

 

My intent:

 

I have purchased 6 x 4TB SAS hard drives and 2x 1TB SATA SSD Drives and would like to move everything from Server 2 into Server 1 so I can decommission server 2 and use my unraid license/key in the future as an off-site backup machine. 

 

That all said above; what is the best way for me to make this transition? Where the docker containers on server 1 are rather small; should I just nuke everything start over; and go from there? Or is there a way I can transition everything over and also be able to use the SSD drives as cache/VM drives over the SAS HDD's?

 

-Cheers,

[Frank1940]

I have just completed moving 19TB of data from my Media Server to the my recently expanded Test Bed server as a backup for my Media files.  I move the data in small chunks (sub directories) to allow unrestricted use of the Media Server.   It took about sixty hours of actual data-moving time on about  four days. 

 

First, I mounted the Media directory on the Media Server on my Test bed server as a remote share using the Unassigned devices plugin.  I didn't have any problems getting the remote share set up.  (There is a manual in the support thread for the Unassigned Devices plugin if you have problems.)

 

Second, I started a session of   tmux   in the GUI terminal so that I did not have to have a Terminal screen always running.  (Google for    Linux tmux   to find all the information you could ever want on tmux!)  (You could also install and use   screen    but if you have the Unassigned Devices plugin installed, you already  have tmux!)

 

Third, I used   rsync  to do the actual copy operation.  (I know. I know!    rsync looks scary at first galance but a lot of the 'switches' are never needed and many actually perform the same task.)  Here is the basic command that I used:

 

rsync -avhPX  source_directory/  destination_directory/

 

Notice the trialing slash---   '/'   ---on the source_directory.   That slash does different things if it is present or not present.  Let me explain with a couple of examples:

----------

rsync -avhPX /mnt/remotes/192.168.1.245_Media/All\ Movies/0-9 /mnt/user/BackupMedia/All\ Movies/

In this case (no trailing slash on the source directory), a directory named  0-9  will be created in the All Movies directory on the destination server and all of the contents of 0-9 directory will be copied into it. 

 

This was the basic command that I used to copy over the 19 directories of media from my Media server.  It created each of the these directories in the All Movies share on my Test bed server and copied everything in each directory.

 

Copying the smallest sub-directory first (which only took about an hour) allowed me to verify that I had all of the switches and parameters correct before I started on the larger ones.

------------------

rsync -avhPX /mnt/remotes/192.168.1.245_Media/All\ Movies/ /mnt/user/BackupMedia/All\ Movies/

Notice the trailing slash in this case on the source directory.  In this case, only the contents of the source directory will be copied over.  So all of the sub-directories will be copied over including the 0-9 one that I used in the first command.  (IF I were to 'forget' that final 'slash' on the source directory, a sub-directory named All Movies would be created with the  All Movies sub-directory on my Test Bed server and all of the sub-directory and all files would be copied also!!)

 

One thing to realize is that rsync is designed as a backup program.  What the above command does is to update the files on the destination to match those on the source.  I added three new movies just today and ran the above command.  It transferred only those three movies to the Test Bed server in about fifteen minutes. 

----------------

 

Another comment, The shell program used in the GUI terminal (named 'bash') has an editor built into it.  So do some playing around with it using the 'normal' navigation keys on your keyboard.   bash remembers previous commands and you can recall them with the <up-arrow> and <down-arrow>.  You can move back and forth within the command with the <left-arrow> and <right-arrow> keys.  You could also google bash...

 

 

 

 

[Frank1940]

A tmux Cheat sheet----

 

osdc_cheatsheet-tmux-2021.6.25.pdf

[dboonthego]

4 hours ago, Frank1940 said:

What the above command does is to update the files on the destination to match those on the source.

 

Those are the same switches I use, except for 'h' which is for human readable output.  The command is more of a delta copy though.  If the intent is to match destination with source, adding --delete will remove files that exist on the destination, but don't exist on the source.

[Frank1940]

12 minutes ago, dboonthego said:

 

Those are the same switches I use, except for 'h' which is for human readable output.  The command is more of a delta copy though.  If the intent is to match destination with source, adding --delete will remove files that exist on the destination, but don't exist on the source.

 

Thanks for your input.  I use the -h switch because I hate dealing with bytes and would rather to looking at KB, MB, GB and TB!   The choice is, obviously, a personal one.  I was aware of the --delete switch but I would rather leave any 'orphaned' files remain in the backup and deal with them as the conditions dictated.  I plan on using the second command line once or twice a month to keep the backup current. 

[resentedpoet]

Thanks @Frank1940 and @dboonthego for all the insight and help. 

 

The insight into rsync and tmux is good to have; never heard of these two tools before. A bit intimidating being command line based but definitely something I will need to look into more in the future. 

 

I ended up going down a different route after adding my 6 HDD's to my server and used unbalance to move everything from the two 300GB drives over to the 24TB array and new CACHE drives. From there, I just SMB'ed over the data from the old array to the new one. I still have to do the docker containers and move them over; but it's not my first time doing that with some of the servers so I think i'll be ok in the process. Will find out soon enough. 

 

Again thanks for all your help. Not to solve some networking issues i'm having with multiple NIC's.