I need to transfer 40TB to my new array and I'm wondering what'st he fastest way.  The most obvious thing is to keep parity unassigned which I have.  My disks (10TB WD Golds) are capable of 250MB/s speeds (and I saw anywhere from 225-250MB/s during the preclear).  The disks (8TB Seagate SMRs) I'm reading from are a bit slower in that they can only do a max of 190MB/s reads so I realize those disks will be my limiting factor.  My network won't be a factor as the servers are connected via 10Gb and iperf3 confirmed the connection between them is 9.25Gbps+.

 

So first off what other settings can I set to ensure the fastest possible copy and second, what utility would be best for this?

Set up disk share and then copy from disk to disk between pairs simultaneously (but never more than 1-1 pair to prevent seeking and fragmentation).

 

If you are doing within the same server, it would be fastest (even with gigabit ethernet, you have to consider network latency).

3 minutes ago, testdasi said:

Set up disk share and then copy from disk to disk between pairs simultaneously (but never more than 1-1 pair to prevent seeking and fragmentation).

 

If you are doing within the same server, it would be fastest (even with gigabit ethernet, you have to consider network latency).

 

Unfortunately disk to disk is not an option because I'm changing my share/split structure.  My data is not equally distributed on my current server and I want to distribute the data equally among my new disks.

If you remove your parity disk on the new server, your speed will be close to the write speed on the target disk. (You will still have the file creation and disk space allocation overhead which will slow things down somewhat.)   You then install the parity disk when everything has been copied to the new server and allow parity to built at that point.  Of course, the new server will not have parity protection until you install the parity disk.  But you already have a good backup... 

On 11/17/2017 at 8:42 PM, IamSpartacus said:

 

Unfortunately disk to disk is not an option because I'm changing my share/split structure.  My data is not equally distributed on my current server and I want to distribute the data equally among my new disks.

 

You still can do disk to disk and redistributing data. Just don't do it in full every time.

e.g. if disk 1 has 100GB and disk 2 has 80GB and you want to redistribute, you can do the following 2 steps:

• 1 -> A 40GB and 2 -> B 30GB
• 1 -> B 60GB and 2 -> A 50GB

 

You can copy from disk to disk in parallel, which really speeds things up. Not sure if 10Gbe would slow you down.

 

Think about ordering and gyrations to maximize parallel operations and minimize more than one read or write from the same disk as the same time.