Hi,

 

I store my documents etc. in a user share totaling ~20GB. I was thinking it would be really nice if I had a backup of all this stuff, and preferably with the following features:

- Runs automatically on the unRaid server

- Runs incremental Time Machine or Shadow Copy style backups

- Allows me to restore a file to a previous state of my choice (hours, days or weeks back)

- ** Bonus** Allows me to manage file restores from the windows file properties 'Previous Versions' tab

 

Does anything like this already exist?

Ideas?

For 20GB, I'd look at some of the online backup services.  Otherwise, I'd just get a quality USB drive and backup to that.

 

~Whip

Crashplan,  I have 400 GB backed up on the cloud

 

Sent from my GT-I9100 using Tapatalk 2

 

 

Hi,

 

I store my documents etc. in a user share totaling ~20GB. I was thinking it would be really nice if I had a backup of all this stuff, and preferably with the following features:

- Runs automatically on the unRaid server

- Runs incremental Time Machine or Shadow Copy style backups

- Allows me to restore a file to a previous state of my choice (hours, days or weeks back)

- ** Bonus** Allows me to manage file restores from the windows file properties 'Previous Versions' tab

 

Does anything like this already exist?

 

I am using some easy linux commands to do this on-server, this means that critical data will copied to a time-machine like setup ON the server (this will prevent issues with multiple drive failure but will not help with catastrophic failure of the whole machine (like fire in your house). Up-side is that it is easy to setup:

 

I have created three scripts in the folder /config of my flashdrive:

 

Users_LastMonth

Users_LastWeek

Users_Yesterday

 

I have created three user-shares each limited to a different (one) physical disk, and each on a seperate disk, my main userdocuments disk is on yet another disk.

 

I list the contents of the Users_Yesterday script , the rest is easy to extrapolate from this one:

 

#!/bin/bash
rsync -avz /mnt/user/users /mnt/user/Users_Yesterday
touch -am /mnt/user/Users_Yesterday/users

 

Then I have added the following lines to my GO file:

 

cp /boot/config/Users_Yesterday /etc/cron.daily/Users_Yesterday
chmod +x /etc/cron.daily/Users_Yesterday

 

The GO rules just make sure that the Users_Yesterday script gets copied to the cron.daily directory, this will effectively make sure it gets executed on a daily basis.

 

The first line in the Users_Yesterday script actually provide the backup to the other disk, the way it is setup it will never delete a file in the Users_Yesterday folder. So a file that got deleted will allways still exist in the Users_Yesterday folder (it also means the Users_Yesterday folder slowly grows, but this is so slow it is not noticeable against the growth of media files).

 

Whenever a file gets CHANGED then on the next day the file in the Users_Yesterday folder will get replaced, this is why I have also created a Users_Weekly and a Users_Monthly variant. Result is that I can:

 

- Allways can back up to yesterdays version of a file

- Allways can back up to the version of last week (I think it performs this on sundays, not sure)

- Allways can back up to the version of last month (copied on the first of month).

 

The second line in the users Users_Yesterday script (touch) makes sure that the date of the "user" folder within the Users_Yesterday folder gets reset to the moment the script ran last (effectively telling me from what moment the last backup is).

 

To protect myself against stupid mistakes I have made sure that the Users_Yesterday folder does not get exported as a share, therefor no chance that I might be working on -it- by mistake.

 

The overall vision I have on backups is:

 

Layer 1: Use good drives and thoroughly preclear (protecting all data)

Layer 2: Use a parity drive, preventing against single disk failure (protecting all data)

Layer 3: Use these scripts to protect my most valuable data against multiple drive failure (protecting my most valuable data)

Layer 4: About twice a year I use some older disks to backup my MY_DOCUMENTS and PHOTOS folders and place those disks in my parents house, preventing against catastropic loss of my whole system or house (protecting my most valuable data).

 

It will never be 100% but this is pretty damn close and it does not cost me a lot in money or time to maintain.

 

 

Helmonder, that definitely looks workable. We agree on the overall strategy as well.

 

Unless anyone has a clue on something more advanced with automatic consolidation or the bonus portion of my requirements (restore previous versions from windows properties) then I do believe I will go implement the same as you have. Thanks!

20GB is relatively small. I use free Cloud Storage for my daily backup.

 

Long-term, larger files go to a Microsoft SkyDrive. I have 25GB there that I got for free.

 

Daily backup goes to a 5GB Google Drive (also free). I use SyncBack SE (free) to sync data from my unRAID server to a local drive on the laptop which then automatically gets synced to the Google Drive.

It's not all running on my unRAID server, but it works and for small amounts of data, I'm happy with it.

 

In SyncBack, I set it to zip each file and password protect as well. This is to save space and also because I do not trust the security of any of those companies!

If its going to "the cloud" make very sure it's encrypted especially if it's personal documents. The Govt has already tried to assert in the Megaupload case that users lose their privacy rights when they use cloud services and are spelunking user data they have captured in that case. If you can figure out how to upload to a Truecrypt volume I would most certainly add that to the mix!

 

There is at least one service that aggregates the various "free" cloud services - skydrive, drop box, etc. that might be worth looking at. I'll try to dig up the name of that I one. Not sure how easily it would be to integrate and I don't think volume crypto would work but it would make for more space :-)

Agreed about the security concerns with cloud storage. I've thought about it and I would be down with the whole truecrupt volume in the cloud thing as an extra precaution if it could be done automatically.

 

But in practice there are some obstacles. The free 50 Gb Box.com I have has a max filesize of only 100MB (and I assume the others also have at least some limitation on max filesize). So to use that I would need to first create a truecrypt volume, then split that into multiple 100MB files, and then upload. An additional disadvantage is if you only have just enough room on your cloud storage then you would have to delete the old dataset before uploading the new updated one, and since it will take a while to upload 5-20GB for most of us, there would be a significant brownout-period of several days where the data is not off-site protected.

 

For now, I keep a truecrypt encrypted 2TB drive offsite that I update once or twice a year, and it holds not only this documents data but also my 1TB of family and semi-pro digital photos and my MP3 collection.

 

If anyone wants free 50GB of Box (with the 100MB max file size), hook up with someone who owns a PlayBook and have them create an account for you (or just log into your existing account from their playbook) before then end of December this year. They've had similar promotions for other devices, not sure if any of those are still running, though. If anybody has tips on where to get big cloud storage for free with larger max filesize limits, I'd appreciate a tip.

 

Back to unraid automation. What would be the next step up from rsync for backup? Surely there's got to be something written for linux that will create incremental backups in archives? With automatic consolidation even? Am I dreaming too big here?

 

Back to unraid automation. What would be the next step up from rsync for backup? Surely there's got to be something written for linux that will create incremental backups in archives? With automatic consolidation even? Am I dreaming too big here?

 

Have you looked at rsnapshot (effectively a perl based front-end to rsync) ? It might meet some of your requirements. It's been discussed a few times in these forums. I set it up a month ago, and I am happy with the results backing up to an external drive mounted in a USB3 dock (the dock allows me to easily periodically rotate a drive offsite).  See also http://www.rsnapshot.org/

 

If its going to "the cloud" make very sure it's encrypted

Erm... yeah. Per my earlier post, I encrypt everything. For the collections of files I even encrypt the file names.

Agreed about the security concerns with cloud storage. I've thought about it and I would be down with the whole truecrupt volume in the cloud thing as an extra precaution if it could be done automatically.

 

But in practice there are some obstacles. The free 50 Gb Box.com I have has a max filesize of only 100MB (and I assume the others also have at least some limitation on max filesize).

 

The max file size for MS Skydrive is 2GB. That, combined with 25GB for free is very useful to me. For the large folders, I rar (and encrypt) with a volume size of 1,900MB, meaning I only need to deal with 2 files for a 3GB folder.

 

If you want a large amount of cloud storage, you're going to need to pay for it. I like the look of BackBlaze, but I don't really have the need to backup everything -- if I lose media files, it's not the end of the world.