Moving to a new server.


Recommended Posts

I just built a second server and I am going to be moving all my data to the new server.  I need to keep my current server up and running while I migrate to the new server.  What would be the best way to move my data from the old server to the new server?  The servers are both connected using 10GB Ethernet and I don't mind the time to move that data over as long as I can keep the old server up and running til the new server is able to take over.

Link to comment

Are you planning to keep the old server with its disks as a backup or something? If not you can just migrate the actual disk drives to the new build rather than copying everything over. But if they are smaller than you would like in the new system or they are using ReiserFS then it probably does make sense to copy them.

 

Of course, you should have a backup of anything important and irreplaceable anyway, whether the older server or elsewhere.

 

If you do want to copy them, you could use the Unassigned Devices plugin on the new server to mount the user shares (or even disks) from the old server, then use command line or mc (Midnight Commander) or Krusader docker (with appropriate mappings).

 

You could also rsync, but if you do that with the user shares my experience is it will create all the folders first and then copy the files, with the possible result that all the files won't fit on the disks in the folders that have already been created. You can work around that in various ways, or just rsync the disks instead of the user shares.

 

There are probably some dockers that might do the job of syncing the 2 servers also.

Link to comment

I am keeping almost all the disks in the old server and would like to copy over everything using the network.  I would like to move everything using user shares instead of rsync the disks because all my data is mixed up right now because I used unbalance to clear my 6TB disks from the server.  I would like to move things over user share by user share.  If you know of a plugin or docker that would make this easier that would be great.  I know that I could just use a windows machine and copy over the network but my switch only has 2 10 GBe ports on it so it would only transfer and 1 GBe speeds.

 

Basically long story long

My old server has 9 data drives with 6 of them 3TB drives and 3 6TB drives.  I built a new server and I am using only 6TB drives in it.  I need to move things over in a way to keep my old server running until my new server is ready to take over because I do not have the time to move everything over all at once if it doesn't work.  If I knew that it was going to go smoothly I would just have swapped out the hardware and restarted the server.  But in my experience things don't always go smoothly and I can't have my server down more than a day.  Once I have the new server up and running I plan on selling the old one with all my 3 TB drives.

 

My new server is going to have 7 6TB drive including 2 parity drives with room to put 5 more drives.

Link to comment

Probably you already know this. A user share is simply the aggregate of all top level folders on cache and array with the same name as the user share.

 

So you could share the disks from the old server, and use Unassigned Devices to mount those on the new server. Then use Midnight Commander to work with the top level folders (user shares) on a disk-by-disk basis.

 

That is probably what I would do since I already know how to do all that and it would be as fast as any other method in transferring the data.

 

The main drawback is it would require you to do some things manually instead of maybe doing it mostly automatically, but since you will only do it once, it may not be worth automating or figuring out what the best automatic solution would be.

Link to comment
23 minutes ago, sgibbers17 said:

Can't Unassigned devices mount network shares?  I am not home now and check but I could mount the user shares using unassigned devices on one of the servers then copy it by user share.

Yes you could work at the whole share level instead of at the disk level. I was just trying to get finer control of the process in case there was a problem with disk space on the new server when trying to do a whole user share at once. I'm not entirely sure how the copy does its thing but I know I had an issue trying to use rsync as I mentioned

 

I was trying to copy a user share that spanned disks to my backup server. Rsync created all the folders first, so they all wound up on one disk, which was OK according to the allocation method (high-water) since the disk was empty and those newly created empty folders didn't take enough space to make them go to another disk. Then it tried to put all the files in those folders that it had already created.

 

Not clear if allocation method 'most-free' would have made any difference that mattered since even if the folders were created on different disks it's uncertain whether all their files would have fit on the disk their folder was on. The way I worked around that was to just move some already copied folders to other disks as it was still doing the rsync to make sure it always had room to continue.

 

If you use Midnight Commander or command line cp or something similar then maybe you won't encounter that behavior.

Link to comment

I mounted a NFS share using unassigned drives and then tried using unbalance to move data over but unbalance does not detect the NFS share.  Is there way to move data over without having an allocation issue that was mentioned above?  I'd like to just tell the old server to move a share to the new server without using a third PC as a go between as it will slow down the data transfer.

Link to comment

Haven't used Unbalance in a while. I don't know if it supports Unassigned Devices or not.

 

There is no need to get another computer involved and I never suggested that.

 

Midnight Commander (mc at the Unraid command line) is a text-GUI that gives you 2 panes (windows) to work with to manage your files. It has always been built-in to Unraid as long as I have been using it. Google has plenty of info about Midnight Commander. It is what I have always used to work with my files on the server.

 

Many people use the Krusader docker. Krusader is a file manager used on Linux GUI desktops. You would have to modify the default mappings for the Krusader docker to get it to work with an Unassigned Device.

 

And of course there is the linux command line.

 

As for the issue I was talking about, it shouldn't be an issue if you don't try to transfer more in one operation than will fit on a single drive.

Link to comment

I brought up using the other PC because you mentioned that rsync had a pre allocation issue when using shares larger than the disk and you were not sure if MC would have the same issue.  I have two maybe three shares that are larger than 6TB so I would have to manually manage what goes to each drive.

 

I just had the idea that if I use the windows VM to make the transfer I should be able to move the files at 10 GBe speeds.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.