Hi All,

 

I recently registered my UnRAID server and have been busy moving several TB of data into it.  I'm quite happy with the software.

 

I've been copying data to the server through a user share and have discovered that it has spanned a couple of drives.  I would rather it occupy just one of my drives.  Oops.  I guess I need to copy data over to another drive.  Not a big deal.

 

The only catch is, being the paranoid sort that I am, all my copies employ data verification.

 

I'm an old-school DOS guy and my tool of choice is xxcopy -- for those who don't know, it's basically the dos XCOPY command on sterioids.  It is capable of, effectively, moving files by copying a file to its destination, doing a byte-for-byte verify, then deleting the source file.  It's awfully handy when moving files from one of my windows boxes.

 

So, through my windows box, I can map one drive to the source disk (lets say drive s:), another to the destination (lets day drive d:), and do the following:

 

xxcopy s:\myshare\*.* d:\mynewshare\*.* /s/e/v2/h/k/rcy

 

and it will effectively move all the files from the source to the destination

 

So my question is, is there a utility or script that will do the same thing from the linux command line on the server.  I'm currently doing it through my windows box attached to the server but I'm sure it would be much faster if I can eliminate all the network activity.  I've seen a few threads here basically doing what I'm doing with the cp and mv comamnds.  it's the verification thing that I'm hung up about.

 

For that matter, am I too hung up on the file verification thing?  Keep in mind this is data that is already on the server so it's already verified and protected through the parity drive.

 

I'm a linux newbie but I've been at this sort of thing long enough that I'm not afraid of a command line. 

 

Any thoughts on this would be welcome.

 

Thanks,

      --Peter

 

 

In all your years of using xxcopy have you ever had it find that a copy error occurred?  (Not trying to be a smart a$$, I am truly interested).

 

I have, from time to time, used md5 checksum computations.  There are cross platform tools that enable you to compute on one platform and verify on another.

 

It would certainly be possible to use your xxcopy to copy from one mapped disk to anohter mapped disk via a Windows machine.

In all your years of using xxcopy have you ever had it find that a copy error occurred?  (Not trying to be a smart a$$, I am truly interested).

 

Strangely enough, it has happened.

 

Generally, it's because a write happens to a bad sector.  I've seen the DOS (this means Windows, too!) copy command happily write a file to damaged media and not throw a fit.

 

But you know, I'm currently doing it through one of my windows boxes and the throughput is actually better than I expected.  I'm getting throughput comparable to what I'm getting when I copy from a hard drive on that windows box. The reads from the network aren't really slowing things down as I thought they woud. 

 

Hmmm.  I'm only running 100mbit on the LAN although both boxes have gigabit Ethernet in them.  Maybe I'll get a cheapie gigabit Ethernet switch and see what happens.  Maybe this will justify upgrading to a gigabit lan.   

 

--Peter

 

In Windows I usually use robocopy, but was looking for the same validation you were when transferring files to an unfamiliar system.  I found the command line version of rsync works very well and has a CRC verification option.  If you are on a 100 Meg network though, it is going to take a VERY long time.  Also, note that rsync appears to be much slower than robocopy at anything I tried to use it for.

 

The only real difference between robocopy and xxcopy in this application is that xxcopy does brute-force byte for byte verification of the copy before deleting the original.

 

However, your comment ftp222 about still running a 100mb network got me thinking.  Since the server has Gb ethernet and the dell box I'm using has Gb ethernet, I just hooked them together and did some tests.

 

Holy cow!  I didn't expect _that_ much of a speed improvement!  I'm seeing a 10x speed increase!

 

I know that gigabit ethernet is theoretically 10x faster, but in real world applications, you rarely need nor see that kind of throughput.  I was quite happy with what I had before and rarely ever needed to push large amounts of data around my home LAN.

 

So, one $35 gigabit switch later, I've copied over about 2 TB of data in relatively short order and all is well.

 

Thanks, everyone.

    --Peter

 

 

Terracopy does as well.

If you use RoboCopy, you should check out RichCopy (another tool out of Microsoft). Its RoboCopy on "steroids" as some say.

I tried richcopy and failed to get it to work, I tried using the drive shortcut I had mapped, navigated directly to the share on network side, directly to disk8, also tried copying to a test folder made on disk8, kept getting system cannot find the path specified error. I couldnt figure out the problem.

I tried richcopy and failed to get it to work, I tried using the drive shortcut I had mapped, navigated directly to the share on network side, directly to disk8, also tried copying to a test folder made on disk8, kept getting system cannot find the path specified error. I couldnt figure out the problem.

 

Keep trying cause you are definitely doing something wrong. You may have possibly typed the wrong path. You should open the paths in windows explorer to verify that you have the correct paths. Then copy and paste them in the corresponding source\destination path in RichCopy.