fast backup mechanism

[rdnkjdi]

I a looking for a fast way to backup to unraid.  We got a limebox and maxed it out.  I'm trying to backup about 5TB, or 3,000,000 images to the limebox, and run an incrimental backup each night to backup anything new, and overwrite anything that has changed.

 

This takes days to do...is there any type of backup program out there that compares directory sizes, and backs up based off of that vs doing a full file scan? 

[WeeboTech]

What backup program are you currently using?

 

I like rsync for file/directory based matching and backup.

 

There is also a tool called rdiff-backup, although I have not used it with unRAID.

[RobJ]

Two more good choices are Unison and SyncBackSE (or the free SyncBack).  I only have experience with SyncBackSE, and can highly recommend it.

[WeeboTech]

Some links for Unison.

(I've never used it)

 

http://www.seas.upenn.edu/~bcpierce/unison//index.html

http://www.linuxjournal.com/content/file-synchronization-unison

 

I think there is a slackware package available for unison.

 

[rdnkjdi]

What program are you currently using

 

Currently trying robocopy...but it takes days.

 

I'm trying SyncBackSE, I think my bottleneck is scanning the lime machine.  Since my files should never change on my backup box, the fast backup from SyncBackSE might speed it up.  I'll try it.  Thanks for the recommendations.

[WeeboTech]

One of the things I found interesting about the unison is that it keeps a local database of the file's modification time and inode. (both remote and local).  If any of those change, then it's a sure sign that the file has changed.

 

This in turn may speed up the need to scan the filesystem

[NAS]

I use rdiff-backup. Worth looking at for a simple option

[subwars]

allways sync is another one