Hi there. I've got my parity and data drives on the way for my new unRAID 6 server and since I'll have the capacity to do it in one shot I'm thinking this might be a good time to transfer everything off my old unRAID 5 server, upgrade it to 6 and reformat those drives from Reiserfs to XFS and then transfer everything from my Flexraid server to back to the original unRAID server.  (The flexraid server is being repurposed as a desktop and once the drives are empty they'll go into the new server for future growth). 

 

If I decide to do this, what's the best/fastest way to transfer ~12TB between the 2 unRAID servers and then ~10TB from Flexraid (Windows) to unRAID?  I've always just used syncback (free) in the past but never for moving this much data before. Also any idea how long should I expect it to take to preclear 6TB drives (3 passes)?  Thanks.

 

One option is to use rsync to transfer between our unraid servers. Assuming you have a Gbit network it would still take a number of days to do this! Syncback would take at least the twice the time since everything needs to go through your PC and back again… Another tool to us could be TeraCopy if you still would like to use your PC and you probably will for the Flexraid server unless you can use rsync on that server as well?

 

When I did a preclear of my 6 TB drives (3 passes) it took 100 hours. You can preclear several drives at the same time but the bandwidth of disk controller might influence the speed as well disk speed.

 

One option is to use rsync to transfer between our unraid servers. Assuming you have a Gbit network it would still take a number of days to do this! Syncback would take at least the twice the time since everything needs to go through your PC and back again… Another tool to us could be TeraCopy if you still would like to use your PC and you probably will for the Flexraid server unless you can use rsync on that server as well?

 

When I did a preclear of my 6 TB drives (3 passes) it took 100 hours. You can preclear several drives at the same time but the bandwidth of disk controller might influence the speed as well disk speed.

 

Good info. Thanks.  Haven't looked into rsync but I'll check that out.  And I guess BTSync would work too now that I think of it.  I just set up the docker app on the unRAID6 server and was thinking it wouldn't work on the other since it's v5 but I see now there's a BTSync plugin too which I assume I could use on that end?  Same idea: get the backup going directly between the servers without a PC in the way.  As for the Windows to unRAID move, I hadn't thought about Teracopy even though I use it as my default for moving individual files/folders via drag and drop.  Maybe I'll just run a test to compary copy speeds between that, syncback and BTSync and figure it out from that.

 

The hardest part of this is going to be persuading the wife to allow the new server into the living room with the other 2 so they can all be GigE connected through the router until everything is moved.  Right now the new server is in my office which is connected via powerline adapter and I don't even want to imagine how long it would take to transfer 12TB through that. 

 

As for the preclear time, 100 hours is better than I was expecting.  It's been a couple years so but I thought I rememberd it taking about that long to preclear 3TB drives on my unRAID5 server (the drives in both cases connected via SATA 6gbs ports on the mb).  So I'd be happy with that since I was thinking that 3 passes might take even a week or so.

 

Thanks again for the input.

Have a look here:

http://lime-technology.com/forum/index.php?topic=38554.msg358683#msg358683

I use rsync for backup and you can probably get some hits on how to do it if you would go that route.

 

I've always just used syncback (free) in the past but never for moving this much data before. Also any idea how long should I expect it to take to preclear 6TB drives (3 passes)?

 

Syncback works fine for large transfers => I've used it for over 10TB several times.  I used to also set the verification option (which nearly doubles the time) to confirm the copies were good, although I no longer do that, since I have MD5's for all of my data ... I just do an MD5 verification after the data's copied.

 

I'd expect 3 passes on a 6TB drive to take 5-6 days.