Okay, I'm up and running with my small unRAID (4TB storage + parity).  I feel confident that my data on the drives is protected from an unexpected drive failure.  However, if my house burned down tomorrow, I still loose everything.

 

How are others on this forum backing up their unRAID to prevent such a loss?  I was thinking of just buying an external eSATA enclosure and stick a couple of 2TB drives in and that would cover me for now.  I have offsite storage.  With an unRAID Plus license, can you have parity + 5 data drives + eSATA connector.  Then connect the eSATA connector and just copy the contents of the disc.  Or is there a better way?  And what to I need to be looking at when I get to 10 TB or more?  Tape backup?  What is everyone else doing?  I wold like to be doing at least a monthly backup of some sort for off site storage.

I'm interested in hearing people's suggestions on these questions as well.

 

On a related note, is it possible to have more than one parity drive? For example, as people are getting servers with 20+ drives, the probability of a double failure statistically increases as well. I was wondering if there is a way to have two parity drives, and be tolerant of two failed drives.

On a periodic basis, I rsync my most important files from one server to another.

I was wondering if there is a way to have two parity drives, and be tolerant of two failed drives.

 

Not yet. There has been talk of it, but no commitment towards this popular feature request.

I already own Acronis True Image.  They claim they software will clone a drive with a reiserfs partition.  So in theory, I guess I could take my unRAID array down once a month, clone each drive onto another drive, and then stash those drives away.  Reinsert the drives back into unRAID and boot it back up.  It looks like that might be the cheapest solution.  A 3TB tape backup costs a couple grand for the drive, plus the tapes.  Then I can get a nice multi-drive hard shell carry case to store them in.

Seems like you might be better off building a second unRAID server and rsyncing disk to disk.

At least then you would not be handling the drives trying to image them.

Tape backup is impractical, but a second machine is pretty easy, especially if you are already committed to double the drives.

 

Even if you built a smaller server with the trayless SATA units, you could still backup multiple drives with rsync over a time span rather then touching your main server's drives.

For each disk in unraid, I have an external USB drive with the same size.

I fill the disks manually one after another and each time I copy a file to unraid, I also copy it to the latest USB-drive.

 

This method is not very elegant, but I feel very good with it. Most of the backup is really offline all the time and safe from digital human errors.

 

On my todo-list: bring them off-location

 

 

Critical items (photos, home video, tax reports, etc) are backed up to multiple USB disks a couple of which are kept off site.

 

Favourite DVDs and Music are also mirrored on USB disk, to save reripping if unRAID box dies. Reripping and/or insurance looks after the rest.

I was thinking of that today.  only 1 TB of my drive space so far is allocated for home movies, pictures, etc.  and that isn't filled.  Everything else is iTunes and rips of my DVD's and Blu-rays.  So in theory if I just backed up that 1TB and lost all my music and movies, insurance should technically cover the lost physical discs.  I just loose the time put into ripping them.

 

So I guess I need to decide if that time is worth the cost of a 2nd server, or if I'm better off just copying the 1TB of irreplaceable data once a month.

I was thinking of that today.  only 1 TB of my drive space so far is allocated for home movies, pictures, etc.  and that isn't filled.  Everything else is iTunes and rips of my DVD's and Blu-rays.  So in theory if I just backed up that 1TB and lost all my music and movies, insurance should technically cover the lost physical discs.  I just loose the time put into ripping them.

 

So I guess I need to decide if that time is worth the cost of a 2nd server, or if I'm better off just copying the 1TB of irreplaceable data once a month.

At least copy it once, now that you are thinking of it.  Try to explain to the spouse that all the family photos are gone... her support of your server will not be anywhere near as high.  Don't need a full server.  Just copy to a writable DVD or two.  you might lose a month's photos or so in a really tragedy, but the bulk will be there on the disk.

 

Joe L.

Some time goes in the naming-conventions of the files and the extra-files (posters, .nfo, ...)

If you want to safe these and not the (big) movie-files, a small script like

 

robocopy z:\ c:\mirror\ /S /E /create /XD excluded excluded2
robocopy z:\ c:\mirror\ /S /E /XF *.mpg *.eyetv /XD excluded excluded2

 

First, all files and directories are created with 0 bytes, then all files except the big files are copied.

 

z:\

would be the drive you mapped an unraid-share to,

excluded excluded2

is  list of directories in this share you want to ignore,

c:\mirror\

of cause the local destination and

*.mpg *.eyetv

the list of extensions of the big files.

 

All in a .zip makes a very compact file for a backup of some work.

 

I do the following:

 

• Keep a copy of my critical documents, family photos, music, software, etc., on my local PC
  
• Use SyncBack to copy critical documents, family photos, music, software, etc., to the unRAID server
  
• Use SyncBack to copy critical documents, family photos, music, software, etc., to external USB drive for off-site storage (weekly)
  
• Store DVD movies, TV shows, and other non-critical media on unRAID server only
  

 

Rationale:

 

Our critical files only total approx. 700GB at present. (easily stored on a 1GB or higher hard drive)

 

Or non-critical movie files total approx. 3TB at present, and growing! (more expensive/cumbersome to store on multiple hard drives at the moment)

 

Important family photos, music, documents, etc., are stored in MULTIPLE locations.  These are the things that are irreplaceable to me and my family.  Not only do I have these critical items on multiple PCs throughout the house -  I also have my wife schlep a portable USB drive to and from her office on a weekly basis to keep another set of files stored safely off-site.  She brings the USB drive home weekly, I use SyncBack to refresh the contents, then she brings it back to her office where it sits for another week.

 

Less critical items such as DVD movies, TV shows, etc., can all be rebuilt from the original disks should I ever encounter a catastrophic hardware failure.

 

I would really like to keep multiple copies of our DVD movies... however, at this time it would be cost prohibitive to keep these non-critical files backed up in multiple locations.  The only drawback is that I would have to rebuild the movie collection from the original disks should anything ever happen to the unRAID server.

 

My strategy above may change once larger capacity hard drives become available (and also drop in price) but that's where I am at the moment.

 

 

I recommend Crashplan and uploading to the cloud.

I use crashplan to backup my critical files on my pcs to the server and to backup server stuff that I never want to lose to a friends server (about 20gb).

 

Seems to work great!

I use a package I wrote called arcvback (see http://arcvback.com/arcvback.html), this does a full backup followed by a long run of incremental backups.  Of course the initial full backup takes a lot of time (perhaps days if you have a few terabytes), but then the incrementals take little time to run.  The backup data files are copied onto two "trains" of external drives and one "train" is kept off site. I have about 4TB of data backed up currently, which is spread across about 3 x 1.5TB disks in each "train".  In a typical week I might add 20-30GB of data to the server (mainly recorded TV or DVD images) and the time to backup that is about an hour.

 

To restore a lost file or directory is pretty simple, the backup program has a database that contains the directory and file name information of all versions of all the files in the backup set, you can just browse that to locate the data you want and then it will tell you which backup packages the data is in.  Then just connect the drive(s) that have those packages on them and it will restore the data to whatever directory you choose.  Since old backups are not overwritten you can get back any version of any file that was backed up since you ran the initial backup.  You might think that this is a potentially large waste of space, but for a large media collection very little is ever changed once it is saved, so typically there is only one copy of every media file in each of the two trains.

 

Regards,

 

Stephen

Random thoughts...

 

Family photos:

I make two sets of DVDs and store them offsite.  Someone else commented on the use of USB drives... I personally would concerned about the long term viability of flash memory (as opposed to optical media) when not regularly refreshed.

 

Extra parity drive:

Would love to have it.  But I also know it doesn't matter how many parity drives you have if your server catches on fire or is stolen so I'm not 100% certain a discussion about parity applies in a thread about backups.

 

Super critical information & unRAID in general:

I see unRAID as a great solution to storing large quantities of media (movies, music, etc) as well as an actual backup destination (rather than source of data that needs to be backed up). 

 

If I had a business need (e.g. a database of customer information that needed to be online 24/7) for critical information, for many reasons unRAID would not be my answer.

 

Someone else commented on the use of USB drives... I personally would concerned about the long term viability of flash memory (as opposed to optical media) when not regularly refreshed.

 

No form of storage is perfect.  Flash and hard drives can get corrupted, and the chemicals in optical media break down over time.  Personally, I trust optical media the least because I've had more bad luck with it than with the other formats.  I used to make DVD backups of everything, then a year later I would find that most of the data was corrupted.  Maybe I just had a bad burner or discs, but either way I no longer trust optical media with anything critical.

 

To answer the original question, I basically don't.  I use Crashplan to backup my documents, photos, and videos, but the latter is growing too fast for that to be a feasible solution.  I'll probably start using an external soon.

No form of storage is perfect.  Flash and hard drives can get corrupted, and the chemicals in optical media break down over time.  Personally, I trust optical media the least because I've had more bad luck with it than with the other formats.  I used to make DVD backups of everything, then a year later I would find that most of the data was corrupted.  Maybe I just had a bad burner or discs, but either way I no longer trust optical media with anything critical.

 

I read recently somewhere that SSDs have a shelf life of under a few months if left completely unpowered.  SSD users would rarely run into this scenario because no one would likely pay SSD prices not to use their disks.  Of course, there's a difference between SSD flash cells and USB key flash cells and all this may be net.legend for all I know.

 

Beyond discussions of the different media itself (which has a big impact), I find burners are really touchy; heads go out of alignment over short periods of time and so what ends up happening is you end up making discs that can only be read by that burner (and only for a short while).  IOW, there is a lot of extra work (which shouldn't be necessary IMO) required to ensure optical media will be a reliable storage medium over time. 

 

For optical media, I also generate *.par2 files for the data on a folder basis.  If a disc is completely lost (e.g. file table), it's lost, but for transient read errors, I have that as a recovery method (in addition to the fact I make two discs).  Also, I use different burners for each disc copy and replace the two burners I have in my primary workstation every two years on alternating cycles, and the off-site media is physically stored in sealed plastic containers in climate controlled self-storage.  I also audit older media from time to time.  Yes, I'm anal but I've never lost data. 

 

Like you, I'm really looking for a better long term solution for critical data more because of the time consuming nature of all that.  Online backup seems to have some potential but the price of enterprise storage is so high I wonder how any cloud storage vendor can have a viable business model... any IT director who's worked with a tier 1 storage vendor like EMC or NetApp, or even a tier 2 vendor like Compellant, knows the price per GB of enterprise storage is several orders of magnitude higher than what we pay trolling the Good Deals forum for our HDDs (e.g. try $10,000 per Zeus 120GB SSD... or as EMC calls them: "EFS drives")... and all my thinking partly assumes I have a bidirectional 100Mb bandwidth as a home user, which of course I don't.

 

For now the answer to the OP question for me really is "for most of what I do with unRAID, I don't backup." 

 

As one who has gone through the pain you are mentioning (unraid server died, all files stuck in limbo) I am going with these guys:

 

http://b2.crashplan.com/consumer/store.vtl

 

 

Unlimited online storage, encrypted, uses SSH, can send them the drives and they will store it on their servers. All for $50/yr.

 

Honestly I can afford to lose all my movies, tv shows, and some games. What I can't afford to lose are all those PDFs, books, and documents I've collected over the year. Do I plan to backup all 6TB? No. 1TB maybe. That'll be enough in case this happens again.

 

This is the first I have heard of crashplan.  Why is that more desired than Mozy?  They seem about the same price.  Does it have to do with the ability to backup network drives or something?  You can get the basic version (not the home version) since software is only run on one PC with the mapped drives, right?

The free home version of CrashPlan lets you back up all your computers to eachother.  Since many of us have multiple computers, this is a great way to get easy data redundancy of our most important files.  You can also backup to off site computers in the same way (though of course it is a lot slower).  I recommend the free version for everyone.  The paid-for version I would reserve for only the most paranoid.

The paid-for version I would reserve for only the most paranoid.

 

I represent that comment.

 

I have there family unlimited plan and use it to back up my sisters, my parents, and my own computer to there cloud. I back up the same computers to my unRAID server also.

Yes - the all you can eat online crashplan service is nice.

I backup my multiple pc's to my UnRaid server and to the online crashplan service.

 

One thing nice about crashplan - it runs on UnRaid server.

I dont see any linux support from Mozy.

Well, I had an empty eSATA enclosure and found a 500GB Green drive to stick in it.  I backed up my desktop, laptop, and netbook to drive 1 of my unRAID using True Image.  Drive 1 was also set as the only share drive for my photos, home videos, software backups (apps that back-up their databases on a schedule, like DVD Profiler) and iTunes library.  So that is all that will ever be on drive 1.  I used True Image to then backup all the contents of drive 1 to the external drive, which I dropped into my desk at work so it is off site.  My parents have a class 125 fire-proof vault, so the drive may end up going there, but it would be less accessible to me, so I haven't decided if that level of protection is needed.

 

So if something destroyed all my home PC's today, I would only loose my movie rips, which would be a great loss of time, but all that could be replaced.

 

I only used half the 500 GB and True Image does incremental backups, so I should be able to use that drive for awhile before it fills up.

Is there anything you can do to protect your USB Key?  I know you can backup the contents of your drive (scripts, etc.), but can you backup your license key.

 

Or if my key dies tomorrow, am I just SOL and have to pay for a new license?  Does Lime have any policy like once per year they will assign your key to a new serial number, or two instances for the life of your license, etc.

 

I used to sell a custom software package that used Serial Shield to protect the contents.  However, the key that generated was based off processor serial, HDD serial, video serial, motherboard serial, etc.  If you changed one or two components in your system, it would still work, but change three and the software would revert back to trial mode (at least I didn't kill it on my users).  So my licensing included the ability to get three keys generated (your initial key, plus two additional should your hardware radically change).  I figured that would get my users through at least 2-3 years before they had to buy a new license.  I know that left the door open to users giving or selling copies of the software to other people by falsely reporting an invalid key, but I also didn't want my customers ticked off if they had some type of PC failure a few months in and had to buy a new key.