I'm in the final planning for a move from nas4free to unRAID and I'm thinking about my data.  I could just build the unRAID server, mount the nas4free share in the console and cp (copy) the data across.  But, I have almost 8tb of data and even with a gigbit connection it would take too long.  So, I want to mount a USB drive on nas4free, copy data to it, move the drive to unRAID, mount it there, and copy the data.  I've have to do this multiple times but I think it would still be faster then the network copy.

 

My questions are:

1) am I crazy to think the USB copy would be faster (I realize I have to copy the data twice but I still think it will be faster).

2) how should I format the transfer drive so that I'm sure I can mount it and read/write it under both system?

3) does it matter which system I use to format the drive before the first mount?

 

Thanks a lot, I'm really looking forward to getting off of nas4free and onto unRAID.

 

Richard

 

ps, even before I could get this posted I think I've found the bigger issue.  No matter how I move the data I'm going to hit a write speed moving to unRAID.  With my test system it appears that I can only write at about 11MBs and I'm betting that won't change if I'm moving from USB or across my network.  Is that right?

 

R

 

 

Are you getting 11MB via the network? It sounds like you have a 100Mb connection in the network path.

 

USB 2.0 will be slower than transferring via 1000Mb network.

Just gut feel -- but I can't imagine copying the data twice -- once onto a USB drive, and once from USB to unraid is faster than a direct copy.  And honestly,  if you aren't in a hurry -- I'd even consider doing it with a tool that verifies it on the target {eg, personally, I'd just use "terracopy" on a windows box to do it, and set it to verify after copy}

 

Since you have the data in a place already, and after a copy, you'd still have the original for awhile -- you can gain some speed improvement by not using a parity drive until AFTER you copy all your data over.   

 

And lastly, many people get about 40mbs - so your 11mbs sounds slow.

 

And while you didn't mention it -- I'll ask if you have done the "pre-clear" on the unRaid drives with several passes -- many (including me) will not use a drive with unRaid until it has been through 3-4 passes with everything OK.  I learned the hard way in building my original unRaid that re-using drives, or even new ones, aren't to be trusted in unRaid yet.  Remember, if you have to go to parity to rebuild from a drive failure - not only are you relying on the parity drive being 100% good -- you're also relying on EVERY drive in the system being good to rebuild ONE failed drive.

Are you getting 11MB via the network? It sounds like you have a 100Mb connection in the network path.

 

USB 2.0 will be slower than transferring via 1000Mb network.

 

yup, I get spikes of 40-60MB and then it drops to about 11-12MB.  USB should give me around 40MB but then I have to do it twice. 

 

The final upside of doing the USB route is that I don't have to fully build a second server.  My ultimate goal is to use the box that is currently running nas4Free.

 

 

the initial population of data should be done w/o a parity drive assigned to speed up writes.

the initial population of data should be done w/o a parity drive assigned to speed up writes.

 

This is a bad idea. Without a parity drive in place there is no way to recover a read failure. Take your time, follow correct and safe procedures and you'll avoid the problems that unRAID was designed to defeat.

 

If your only getting 11-12MB on a Gigabit network there are problems that need to be resolved. Using USB to work around network problems does not seem productive. These issues need to be resolved regardless of how the server is loaded with data.

 

Post a syslog taken after a slow transfer occurs. Post SMART reports for the drives. Show the output of "ifconfig" and "ethtool eth0"

the initial population of data should be done w/o a parity drive assigned to speed up writes.

But only if you also use a transfer method that verifies the copy. 

 

Otherwise you are likely to have corruption if you hit an un-readable sector on a disk.  If the transfer is limited by the network, and if you are just going to start a copy and let it run, then many of us advise users to put the parity drive in place first, and then copy the data.  The resulting data is then protected from the start and it allows unRAId and the disk's SMART firmware to automatically deal with un-readable disk sectors by re-allocating the sectors for you.

the initial population of data should be done w/o a parity drive assigned to speed up writes.

If your goal is a temporary speed boost, I agree, as long as you keep your data on the old drives until the new system is fully up with parity built and a good post build parity check is completed. Then you need to spot check data integrity against your originals to make sure your data didn't get clobbered during the transfer.

 

My personal thought is that the initial time saved is not worth the risk, and wondering if your data is really ok.

 

Each person has their own risk tolerance, and needs to weigh the options for themselves.

indeed.

 

Copy.

Verify.

Delete.

 

DO NOT MOVE data.

Are you using a cache drive? You should be getting way faster speeds than that if you are. Even without one though you should still see a lot faster than 11MB/s unless you're moving, say, hundreds of thousands of very small files. For videos and such you should see around 25-35 sustained writing directly to the array. I would verify that all your devices are actually connecting at 1000Mbps.

Are you using a cache drive? You should be getting way faster speeds than that if you are. Even without one though you should still see a lot faster than 11MB/s unless you're moving, say, hundreds of thousands of very small files. For videos and such you should see around 25-35 sustained writing directly to the array. I would verify that all your devices are actually connecting at 1000Mbps.

11 MB/s is exactly what I would expect if either network connection (or the router/switch in between) was at 100Mb/s.