I'm looking at doing a full backup of my unRAID server to my externals. How often should one do these? Is reading out 4TB+ out of HDDs bad for them?

 

I'm backing up from a Windows PC and would like incremental and occasional (monthly) full backups. The data is mostly static, gets added and then never moves. Data is rarely deleted. If anyone wants to recommend something, that'd be a bonus.

My first question... Where would you "backup" the drives too?

I believe he means he would like to backup his unRaid server to external HDD's attached to his Windows machine.

Dirtysanchez is correct.

No, it does not "hurt" your drives in any way to read the data !  That's what they're designed for 

 

As for a process to do the backups ... I outlined what I do in paragraph 5 of this writeup I did on backups:  http://lime-technology.com/forum/index.php?topic=31020.0

 

Hi garycase,

 

I don't see any "windows" solutions posted, I have external hard-disks to do this but no additional server (yet). What would you advise for incremental backups?

I do everything from Windows -- did you read the process I use (in paragraph 5) ??

 

Sure did!

 

From what I understand, you manually copy the new files over. I don't really see any new mentions of specific software, or is this all done by hand?

I do it all manually -- whenever I copy a file to the server, I copy it to the current backup disk.

 

When that disk gets full; I simply pop in a new one and continue the process.  With modern drive sizes, I don't need to change more than once/year (or even longer).

 

Note that I also have a backup server that I power-up once/month and then synchronize with my 2 UnRAID servers using SyncBack profiles  (this whole process is automatic via a scheduled process that turns on the server and runs the profiles).

 

Since you apparently haven't been backing up routinely, you'd obviously need to manually copy everything from the server initially -- but that's VERY simple ... just do a "Copy" or, if you'd prefer, install SyncBack, SyncToy, or any other synchronization utility you like and use a profile to copy to your first set of backup disks.  Then just ensure you keep things current.

 

Ah yes, I understand that much... the major problem is I also want INCREMENTAL backups. i.e: I want to maintain changes in case of files going missing etc. How do people manage this kind of thing? If you just blindly copy everything, won't you end up with duplicate data, data missing and files gone stray? That's why I thought incremental backups would help me.

If you want automated incremental backups, you'll need to set up a backup server instead of using individual external units.    Then you can simply use something like SyncBack and it will be completely automated -- and always up-to-date as of the last run of the profile.

 

I do that with my backup server; but not because I'm concerned about my backup disks ... just because it' more convenient.    But I still maintain my backup disks -- I store the full ones in a waterproof, fireproof, data-rated safe, so they're my real "catastrophe" backup.

 

As for ending up with duplicates, etc. => Yes, if I replace a file, I'll have that stored on 2 different disks [if I replace it before I finish filling the backup disk it was originally stored on, the newer file will replace the older one and it won't be an issue].    But if I need to restore it, I'd simply use the copy on the most current backup disk.    I have a complete directory of all of the backup disks stored in PDF files, so it's trivial to search for any given file.

 

I see. That makes sense, so the files might end up in multiple places. I guess this can be avelitated by simply doing a "full backup" from time to time. i.e: retire the old backup, get a new one and do a fresh, full copy. Then, the directory contents would be most up to date.

 

That helps a bit, as I'm worried about accidentally deleting files, so duplicating is good I guess, too. That way, everything that ever existed on the drive remains in tact.

 

Looks like Microsoft's SyncToy can do this for me (not spanning multiple disks). This is suffice for now, I'll have to look into also getting a backup unit as well.. just need to find where to put it.

So, I guess this leaves me with one question:

 

If you move a lot of files, in your case, you end up with duplicates everywhere. Won't this make for a messy recovery? i.e: you could end up with a lot of files all over the place

First, I don't "move a lot of files".  Almost all of my media is "static" ... once I've copied a song, movie, etc. to the server there's no reason to move it around.  But even if I did change a lot, it's no big deal.

 

Assume the worst case -- the whole UnRAID box gets zapped and EVERY disk is wiped, so I have to replace them all.  And it happens at the same time the backup server is being updated, and that server also gets completely zapped.  [Needless to say, this is a VERY unlikely scenario -- especially since every one of my PC's has a dedicated high-quality UPS]

 

I would simply rebuild the UnRAID server; then do the following ...

 

(a) Insert Backup Disk01 into a caddy, and run my Syncback Profile to copy everything from it to UnRAID.

 

(b)  Repeat Step (a) for Backup Disks 02, 03, 04, ... (I currently have 17).

 

If a file had been modified several times, it may be on several of the backup disks -- but the most current version will be on the highest-numbered disk that contains it.    So the file will automatically be replaced each time a newer version is encountered.    The end result will be that everything is up-to-date.

 

This sounds much harder than it is.  It takes perhaps 1 minute to insert a disk and run the profile.  Granted the backup profile will run for many hours (the backup disks range from 1.5TB to 4TB) ... so the overall process would take a week or so;  but the "human time" involved would be less than 30 minutes.

 

When I was on Windows, I used SyncBack Pro as well. Here overview of my backup system.

 

I had Incremental backups that ran every day of the week, full refresh once a week. Then I had a full backup once a week on another drive, full clear at every run. And another full backup which was refreshed once a month, culling deleted files every six months. Latter two drives were "offline", ie. not attached to my desktop for security purposes in case there's data corruption or electricity issues (I have an UPS but I'm paranoid ).

 

That way I would have daily backups of all critical data, and multiple backup levels as a fallback. If I accidentally delete a file or notice data corruption, I'd have it handy for up to six months.

 

Going forward, I will be rotating two HDDs on an offsite location (one at home, other at work), which protects me from theft, fire, water damage, etc. Crashplan used to fill that role - I will continue using it, but will also rotate HDDs.

 

And no, doing full backups is not bad for your drives. And if it was, it would be even more important to do backups