SyncBack for Windows


danioj

22 posts in this topic Last Reply

Recommended Posts

Are there many people out there using SyncBack for Windows to Back-Up their Unraid System. The only person I know who is, is garycase but I figure other must be too. The issue I have is the time it takes to prepare the Backup so I figure I must have something set wrong.

 

I am Incrementally Backing up a 13TB Array per week with about 300GB per week additional data. I have the Backup set for weekly. Like I said I have it set to be incremental in that it will not copy existing files, delete or overwrite.

 

HOWEVER it takes over 2 days to read the entire Array and generate the hash values for the Array to decide if there has been a changes before it proceeds to copy the changed files / new files.

 

~2+ days of Array reading per week seems like allot of array effort.

 

Has anyone got any pointers?

Link to post

What are your detailed settings in SyncBack ??

 

Mine is set to simply use file date/time to determine if a file has changed ... NOT to completely read the file and regenerate a hash value.

 

I might have to turn off the hash generation. I think I clicked it because I wanted the copy to verify via a hash value after copy. I think I misunderstood what this option was for ....

Screen_Shot_2015-06-21_at_9_20.17_am.png.6064d016d5c69f021f27d7f0d22e07aa.png

Screen_Shot_2015-06-21_at_9_19.02_am.png.96f60d35659011720f8a25a372944519.png

Link to post

Well, the "slower but more reliable" method of determining differences is indeed SLOWER  :)

 

I've actually never used that setting ... but I'm not surprised that it computes hashes.

 

I DO use verification, so all the files it copies are verified (which basically doubles the backup time) ... but it still doesn't take anywhere near the time you're seeing.

 

Link to post

Well, the "slower but more reliable" method of determining differences is indeed SLOWER  :)

 

I've actually never used that setting ... but I'm not surprised that it computes hashes.

 

I DO use verification, so all the files it copies are verified (which basically doubles the backup time) ... but it still doesn't take anywhere near the time you're seeing.

 

Right - I've removed that setting and have hit Go! Let's see how quick it is.

 

Thanks.

 

P.S. with the features I am using do you see a need for Pro? If I remember correctly you use SE right?

Link to post

Well, the "slower but more reliable" method of determining differences is indeed SLOWER  :)

 

I've actually never used that setting ... but I'm not surprised that it computes hashes.

 

I DO use verification, so all the files it copies are verified (which basically doubles the backup time) ... but it still doesn't take anywhere near the time you're seeing.

 

Right - I've removed that setting and have hit Go! Let's see how quick it is.

 

Thanks.

 

P.S. with the features I am using do you see a need for Pro? If I remember correctly you use SE right?

 

That worked a treat!!! It has done its check and is now Backing up the 800GB difference between the Array's @ 51MB/s :-)

 

Perfect.

Link to post

I doubt there's anything you really need in Pro.  You're correct -- I use SE.  I bought it primarily because of two features:  (1)  it can back up open/locked files; and (2) it HAD much better error reporting than the free version (no longer true ... the free version has improved markedly in that regard).    #1 is still a very nice feature for my Windows backups ... not sure it matters on my UnRAID backups.

 

Link to post

Interesting thread..

 

I might consider purchasing SyncBack.

 

How did you setup the Backup share on unRAID and what are the split level?

 

I don't think that the split level is really material in this discussion. In my scenario I have 2 machines both running unRAID. Shares are mirrored on both machines. In this case:

 

"nas"

 

I basically use an "intermediary" Windows VM running Syncback Pro to Backup one has share to the other

 

With the settings I posted above.

 

As a matter or record I used the Fill Method: Most Free and Split Level 2 BUT that has no bearing on my Backup Strategy!

 

:)

Link to post

Mine is basically the same => I have a SyncBack profile for each share and simply back it up to the backup server.

 

If you're familiar with SyncBack, you create "profiles" that define the source, the destination, and a LOT of options about how you want the backup or sync to work.  It can be a backup only; a synchronization between two locations; and you can choose whether the backup should mirror the source or should retain files that were previously backed up but are no longer on the source.    It's very easy to configure it to do exactly what you want to achieve with the backup.

 

So I simply have one profile for each share, and they're all set up with the same set of options.    As an example, suppose you have shares named Movies, TV Shows, Music, System Backups, and Personal Data.  You would then create backup profiles for each ... perhaps named something like this:

 

Backup Movies

Backup TV Shows

Backup Music

Backup System Backups

Backup Personal Data

 

You can then create a "Group" profile to do it all ... e.g. perhaps name "Backup Now" => and it would include all of the profiles I just listed above.    So in SyncBack you simply point to "Backup Now" and click Run.

 

Or, even easier, you can set SyncBack to run the "Backup Now" profile at a scheduled time on whatever days you want (daily, weekly, monthly, every-other-day, etc. ... whatever schedule you want).

 

You can also create an icon that will load SyncBack and automatically run the profile by simply appending the name of the profile to the end of the command line.

 

My setup runs SyncBack from a separate Windows box that's always on => I believe Daniel has his set up to run from a Windows VM that's on his primary server ... a neat way to do it, since it doesn't require a separate computer.

 

 

Link to post

As I note above, I solved the initial issue which was the driver of this thread. However I have another one now  ;D Given the topic was so broad, it seemed to feel right to keep going with this thread.

 

As can be seen from the attached screen shot. I have a well thought out backup strategy. I take a mirror of the Flash and App Drive (of both Servers) to my Main Server Array Daily, Weekly and Monthly. I back up my Main Server Array to the Backup Server Daily (Incremental Backup, no Delete). No Issues with my strategy.

 

Where I have an issue is that Sync-Back Reports a Failure on the App Drive Backups. This makes sense because it fails on the VM's .vdisk files and the  Docker .img files on both servers. I might add that it shows as a fail quite rightly as I have "Verify Copy" enabled and the files have changed from initial check to copy (as they are fluid live files).

 

I have no real issue, because the file still gets copied and I have my Backup.

 

My question is, can I get Sync-Back to ignore the errors on these files ONLY because of their "Live" nature so I can get it to report a successful run?

Screen_Shot_2015-07-01_at_6_55.30_pm.png.548e86aad197c34ca653e029f2291ab4.png

Link to post

Depends on the reason it's failing ...

 

If it's failing because the file is open then probably not.

 

If the reason for the failure is simply that the verification fails, then you can simply modify those specific profiles to not do a verification.

 

 

Link to post

Depends on the reason it's failing ...

 

If it's failing because the file is open then probably not.

 

If the reason for the failure is simply that the verification fails, then you can simply modify those specific profiles to not do a verification.

 

Thanks.

 

Damn. I thought that would be the case. Yeah, it's failing because the Docker image and VM disk files are open. They never shutdown and I don't want to have to automate periods of the day when the servers go down so they can be backed up.

 

It is not much of an issue because the files are still being copied. It is more my OCD. The Sync-Back job "says" failed, and I don't like it. Given all the options in Sync-Back I thought there might be a switch to ignore errors on particular files. No matter. I guess I'll learn to live with it.  :)

Link to post

One of the features of SE and above (I note you're using Pro) is that it can "... Copy open/locked (in-use) files ..." => but unfortunately that only works with Windows, as it uses VSS to do those copies.

 

I just noticed that there's an option in the Advanced Copy/Delete options that says "Do not use the Volume Shadow Copy service to copy open files."    I do NOT know what that does -- it may simply mean you can't copy locked/open files; or it may use another technique, which could, if that's the case, work in Linux as well.  I simply don't know -- but it's certainly worth a try.

 

 

 

Link to post
  • 2 weeks later...

One of the features of SE and above (I note you're using Pro) is that it can "... Copy open/locked (in-use) files ..." => but unfortunately that only works with Windows, as it uses VSS to do those copies.

 

I just noticed that there's an option in the Advanced Copy/Delete options that says "Do not use the Volume Shadow Copy service to copy open files."    I do NOT know what that does -- it may simply mean you can't copy locked/open files; or it may use another technique, which could, if that's the case, work in Linux as well.  I simply don't know -- but it's certainly worth a try.

 

I just wanted to close this off.

 

I tried your option and it didn't work. I decided to "live with" the fact that the Backup reports a failure as the file still gets copied it just reports that the file copied is different to the one it started with. Which is true as the files which fail are "live" files changing all the time. Docker / VM images etc.

 

Just for my own piece of mind I restored from one of these files and all was good, so the copy operation is fine. Just I get an error.

 

As a side note, my free trial of Sync-Back Pro ended and I had to decide what to do. I downloaded the Free version and imported all my profiles. Turns out no option I had selected was unavailable in the Free version, so I have stuck with that and uninstalled Pro.

 

Excellent Software for Free. Excellent.

Link to post

Agree the free version is excellent.    When I bought SE it was because it had better error reporting and the ability to copy locked files -- but as you've discovered that's only true with Windows.    The free version now has the much-better error reporting that I bought SE for; and the locked files copy capability isn't nearly as useful as I'd anticipated.

 

Bottom line:  I no longer recommend that folks buy SE or Pro for most scenarios => the free version is good enough for the vast majority of use cases.

 

Link to post
  • 3 years later...
  • 3 weeks later...
On 4/11/2019 at 11:01 AM, scubieman said:

Do you guys use this as a docker to backup unraid? or in a windows VM on unraid?

Neither.  I run it on a micro Dell (Inspiron 3050), whose only purpose is to back up my array.  I figure- it uses almost no power, it saves me the headache of setting up a VM (which I've not proven very good at doing), doesn't impact my server's performance, plus it gave my 3050 a purpose...I had the thing, but couldn't decide how to use it.  I have it physically atop my server, sharing a small, dedicated 5-port switch, so that mountain of data doesn't have to move across my entire network during backups.  I start a backup (using remote desktop) then log in periodically. 

Link to post

I also use it on a Windows box --> I have a scheduled task that wakes up at 4:00 in the morning; sends a WOL signal to my backup server; waits 3 minutes to be sure it's had time to boot up; and then runs a SyncBack task to back up my two always-on UnRAID servers to my backup server.   After the task completes, it sends a command to the backup server to shut down.    Actually, since I switched both of my servers to dual parity, I've thought about changing this task to only run once/week … but haven't got around to doing that 😊

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.