tr0910 Posted March 20, 2011 Share Posted March 20, 2011 I have been testing my unraid to death making sure I have a good backup system in place. I had planned on having the unraid boxes backup the workstations keeping the pictures safe. I can either have the unraid pull the files from the workstations, or have the workstations push the files to the unraid box. The following 2 methods compare the exact same 5300 files. unRaid Server Initiated rsync -av --stats --progress /mnt/cs5 //mnt/disk3/Pix2011/ WinXP64 workstation initiated. rsync -rav --stats --progress /cygdrive/d/"my documents"/"My Pictures"/"Pictures 2011"/ //tower1/disk3/Pix2011/ Interestingly, initiating the task from the workstation for a dry run with 5300 files and 130 gb results in about 90 seconds of run time. Initiating the task from unRaid results in a run time of about 15 seconds. This is where no files have changed, and it is simply verifying everything is up to date. Network is wired gig ethernet and running this with several gigs to transfer results in much more even time between the 2 methods. Why is it so much faster initiating from unraid and pulling from the workstation, rather than initiating from the workstation and pushing to the server? Would daemon mode be any faster? Quote Link to comment
Spectrum Posted March 20, 2011 Share Posted March 20, 2011 If I had to guess it would be the difference in recursion (-r) option between the two calls. If that is a typo and both calls are actually recursing, the difference could be added overhead going through the cygwin1.dll for cwrsync. I've never actually done any comparisons between rsync and cwrsync so it would be interesting to see if that is the difference. I have no clue about speed difference using rsync in daemon mode; hopefully someone else has some info on that. Quote Link to comment
tr0910 Posted March 21, 2011 Author Share Posted March 21, 2011 If I had to guess it would be the difference in recursion (-r) option between the two calls. If that is a typo and both calls are actually recursing, the difference could be added overhead going through the cygwin1.dll Yes, that was a typo. Both are recursing. I am blown away at how fast rsync is when run from the unraid box. It sure is a lot slower running via cwrsync. I have the windows task scheduler calling the script that does the backup to unraid, and having it done this way will likely be easier to manage as you suggested. The speed difference noted is interesting, but not terribly significant. I will have the scheduler wait until the workstation is not being used for 10 min before kicking off this event, so it likely won't be noticed. Quote Link to comment
Spectrum Posted March 21, 2011 Share Posted March 21, 2011 15 seconds vs. 90 seconds is interesting. 15 min vs. 90 minutes is problematic. 15 hours vs. 90 hours is nuts. It's in the unit of measurement Quote Link to comment
JarDo Posted March 21, 2011 Share Posted March 21, 2011 I'm using the pull method to backup my server to my unraid. rsync -amv --delete --chmod=Fo-x --log-file=/boot/logs/fs1_rsync.log 192.168.1.90::fs1 /mnt/user/backup/fs1 On a morning where there are no files to backup the log shows that the whole operation takes about 17 minutes to complete: 2011/03/11 04:41:25 [13845] receiving file list 2011/03/11 04:54:34 [13845] done 2011/03/11 04:58:16 [17308] sent 25 bytes received 3594148 bytes 3535.83 bytes/sec 2011/03/11 04:58:16 [17308] total size is 257453877548 speedup is 71630.91 Funny thing is, this is the bulk of time involved. Even when I've got a few gigs of ISO's, or other assorted files, to sync the average backup is between 21 and 30 minutes. Quote Link to comment
tr0910 Posted March 21, 2011 Author Share Posted March 21, 2011 I'm using the pull method to backup my server to my unraid. rsync -amv --delete --chmod=Fo-x --log-file=/boot/logs/fs1_rsync.log 192.168.1.90::fs1 /mnt/user/backup/fs1 On a morning where there are no files to backup the log shows that the whole operation takes about 17 minutes to complete: JarDo, how many files are being dealt with in those 17 minutes? I'm guessing well over 100,000? Quote Link to comment
JarDo Posted March 21, 2011 Share Posted March 21, 2011 It looks like 171,000+ files. So, I suppose 17 minutes isn' too bad. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.