landS Posted July 12, 2013 Share Posted July 12, 2013 My current Main Server is a Win 7 Machine (2500K processor) with 8 HD's (3 TB), each pair in a mirrored array. This is what I initially stored all my legally owned movie collection on. Now my entire collection is copied to my always on unraid server (in my signature) with 1 parity drive. Is there an option to have an unraid machine that I turn on once a month to backup the main unraid machine so that I can do away with the power hungry Win 7 Machine? I can sell the Adaptec SAS controllers and nearly fund an entire 2nd Unraid box. I already have enough disks to re-purpose from the win 7 machine. Thanks! Quote Link to comment
garycase Posted July 12, 2013 Share Posted July 12, 2013 As long as you have another UnRAID license for it, just build a 2nd server with enough storage capacity to backup your server; give it a DIFFERENT ID [Can't have two servers named "Tower" ]; and then just switch it on as desired and run a synchronization utility to do the backup. For example ... If the main server is named "Media", and the backup server is named "MyBackups", and the share you wanted to backup is named "Movies" ... then you could initially do the complete backup with the following command: Copy \\Media\Movies \\MyBackups\Movies If you wanted this to be automatic, you could do the following: (a) Configure the backup server for Wake-on-LAN (b) Set up a scheduled task on either a Windows PC or the main UnRAID server to do the following: -> Send a magic packet to turn on the backup server -> Wait 2 minutes (or whatever time it takes to boot) -> Run a synchronization utility to update your backups -> Send a "Shutdown" command to the backup server I know how to do this in Windows ... you'd need a "Linux guy" to provide details on the script and Cron setup needed to do it in Linux. Quote Link to comment
ZeroK Posted July 12, 2013 Share Posted July 12, 2013 There are lots of ways to back up unraid. If you have two unraid servers then rsync is the best but if you only have an unraid server and a windows computer then garycase's way is correct or this way. This is how I back up my two unraid servers to my windows home server. I use robocopy and scheduler. Robocopy using this command will only copy things that have changed and ignores the rest. It's always done as robocopy <source> <destination> <options> (/MIR makes a mirror image of the source and if it's already there it skips it so backups take less time if you only added one movie) /R:0 (will retry an open file a specific amount of time in case someone has it open) and /W:1 (will wait 1 second before trying again) (without these options if it comes to a file that is open it will wait for user input) robocopy \\TOWER\HD_MOVIES \\BACKUP\HD_MOVIES /MIR /R:0 /W:1 Create a text document, make sure that your file extensions are unhidden and name it whatever you want but put a cmd file extension on it, for instance, backup.cmd Put the above command into it, save it, then test it on some data you don't care about first to make sure it works as you want it, then set it up in the scheduler to run whenever you want. You can have multiple lines in the same file so if you want to backup multiple shares it will start the next share once it finishes with the first command. For instance... robocopy \\TOWER\HD_MOVIES \\BACKUP\HD_MOVIES /MIR /R:0 /W:1 robocopy \\TOWER\TV_SHOWS \\BACKUP\TV_SHOWS /MIR /R:0 /W:1 robocopy \\TOWER\MP3s \\BACKUP\MP3s /MIR /R:0 /W:1 Great thing is, Robocopy comes free with windows. There is an option for multithreaded copying, meaning it will copy multiple files at the same time, but this option usually ends up taking just as much time and saturates the bandwidth, plus it doesn't show a progress bar. Quote Link to comment
landS Posted July 14, 2013 Author Share Posted July 14, 2013 Thanks! What I am really thinking about doing is having the following: Windows Work Machine - For office use, gaming, encoding when new purchases are made. This machine changes frequently and tends to be VERY power hungry. No need to keep hard drives full of storage on this rig. This machine should only have network access for read/write to the file storage location Tower - My main storage repository. Holds ALL my windows backups, work files, music files, movie files Tower2 - a backup of all the storage repository. Holds a copy of all the data should anything happen to the first Tower. **** Originally my main file repository was on my Windows machine on Mirrored disks. Now my main file repository is on my Unraid machine with a single disk fault tolerance. I would like to do away with my Windows machine storage and follow Gary's advice: (a) Configure the backup server for Wake-on-LAN (b) Set up a scheduled task on either a Windows PC or the main UnRAID server to do the following: -> Send a magic packet to turn on the backup server -> Wait 2 minutes (or whatever time it takes to boot) -> Run a synchronization utility to update your backups -> Send a "Shutdown" command to the backup server So now the search is on in how to use rsync, the pluses and the minuses Quote Link to comment
RockDawg Posted July 15, 2013 Share Posted July 15, 2013 Lately I've been considering setting up a second backup unRAID server myself. Would there be any negatives to running both as VM's on the same host? Quote Link to comment
garycase Posted July 15, 2013 Share Posted July 15, 2013 Lately I've been considering setting up a second backup unRAID server myself. Would there be any negatives to running both as VM's on the same host? :) Think about what you're proposing -- remember this is a BACKUP. As in designed to protect you if something happens to your main machine !! Clearly your backup should NOT be on the same machine you're backing up. While this is indeed better than NOT backing up ... many people have a backup disk mounted in the same machine -- it only protects you against a failed disk => not against a major problem with the machine due to a power spike, catastrophic failure of the power supply (which can "take out" a lot of components with it; flood or fire damage; etc. My backups are at the far end of our house ... and serious corporate backups are not only off-site but in a different city. An example of some GOOD backups => there was NO data lost by any of the major financial companies in the World Trade Centers on 9/11 !!! In other words, it's MUCH better to build a low-power backup system and put it somewhere else in your home. Turn it on via WOL, do your backups, and then shut it down. Quote Link to comment
JonathanM Posted July 15, 2013 Share Posted July 15, 2013 In other words, it's MUCH better to build a low-power backup system and put it somewhere else in your home. Turn it on via WOL, do your backups, and then shut it down. If you are only using it for short periods, power consumption isn't a concern. Just throw together the cheapest box you can, use your castoff smaller hard drives, whatever gets you a current backup. Do it sooner rather than later. I deal with people on a regular basis that all say, I was planning on backing it up, but I never got around to it. Data recovery is VERY expensive in some cases, you could put together a very nice 20TB unraid server for the average price of an expedited clean room recovery. Quote Link to comment
RockDawg Posted July 15, 2013 Share Posted July 15, 2013 I didn't consider a power surge or something similar. I run through a UPS, which helps a lot, but I know it won't prevent everything. Off site backup is just not feasible for me so that's out of the question and I don't see a lot of value to putting the backup server on another side of my house except for maybe a fire that is isolated to one area of my house. I know I can't/won't protect against every possible case. Just looking for something to cover most cases and especially data corruption and accidental deletion. Quote Link to comment
garycase Posted July 15, 2013 Share Posted July 15, 2013 I didn't consider a power surge or something similar. I run through a UPS, which helps a lot, but I know it won't prevent everything. Off site backup is just not feasible for me so that's out of the question and I don't see a lot of value to putting the backup server on another side of my house except for maybe a fire that is isolated to one area of my house. I know I can't/won't protect against every possible case. Just looking for something to cover most cases and especially data corruption and accidental deletion. Understand ... my "other side of the house" comment was just to emphasize the "not on the same machine" aspect of it But if you have Cat-6 wired throughout the home, it's not a bad idea to have it in a different room ... most fires don't totally destroy a home; burglaries don't usually involve taking EVERYTHING (so if they take one, they'll likely miss the other); etc. And a major electrical "hit" (e.g. lightning strike) that "takes out" one of your UPS's and damages connected equipment would be unlikely to "take out" a second one as well. But even a backup array on the same physical machine (in a 2nd VM) would indeed protect against data corrupted and accidental deletion ==> it's the same concept as having an extra drive in a system with an automated backup script that backs up to the 2nd drive. The key thing is, as jonathanm noted, to DO IT !! A backup that you "meant to do" provides ZERO protection I agree, of course, that if it's just a backup server and only turned on for the backups, that the power consumption isn't a major concern ... my "low power" comment wasn't mean to mean spend a lot of $$ to keep power consumption low; but to note that you clearly don't need a high-power CPU, 7200rpm drives, etc. that you might want in a primary server. Quote Link to comment
Kryspy Posted December 5, 2013 Share Posted December 5, 2013 What is the easiest way to backup to a external hard drive from unRAID? What file format to use? I also have an iMac. Kryspy *Nevermind....... bit torrent sync seems like a good fit for me at the moment Quote Link to comment
dchamb Posted February 28, 2014 Share Posted February 28, 2014 Well I am also interested in a good backup plan that allows me to take the backup disks offsite easily. The only way I know how to do that is to get some drives that can at least match the capacity of what I am using on the unRaid system (8TB right now), and connect them either through esata or USB 3.0. Now unless I use backup software that can span volumes, I would need to break up the files so that they can fit on the external drives, since I don't know of anyone producing drives greater than 4TB these days. Quote Link to comment
garycase Posted February 28, 2014 Share Posted February 28, 2014 If you want to physically move the backup media off-site, then the simplest approach is to backup to a set of bare drives (using an eSATA or USB dock) and simply store them in WiebeTech DriveBoxes. http://www.overstock.com/Electronics/WiebeTech-DriveBox-3851-0000-11-Hard-Disk-Case/5792263/product.html I do this with one set of backups, and store them in a fireproof, waterproof, data-rated safe. There are several ways to keep them up-to-date: (1) Copy everything you write to your array to a "current backups" disk at the same time; and simply replace that drive (and store it away) when it gets full. This is what I do, since I'm the only one who every writes anything to our array. (2) Backup the entire array to a backup disk, copying only files that have been modified since a specified time. As long as you keep track of the time you do each backup; and then use that as the starting time for the next one, you'll capture all new/changed data with each new backup. (3) Backup the individual UnRAID disks one-on-one with a matching size backup disk. Note that this potentially requires cycling through as many backup disks as you have array disks for each backup cycle ... although depending on your usage it can be far less of a hassle than that. For example, if you use a "fill-up" allocation strategy, then UnRAID only writes to one disk until it's full ... so backups are significantly simplified; although you'd still need to use the technique in #2 to ensure you captured any changed files from other disks. Depending on the amount of data you have, it may be far easier to just use a portable backup server; and carry the server off-site. There are a couple of nice portable mini-ITX cases that can hold 6 drives. With 4TB drives, this would give you 20TB of protected storage ... or 25TB if you wait for the forthcoming 5TB units that both WD and Seagate are expected to ship "any day now" (at least sometime this year). A couple of nice cases for this ... http://www.newegg.com/Product/Product.aspx?Item=N82E16811345027 http://www.newegg.com/Product/Product.aspx?Item=N82E16811112348 Quote Link to comment
tr0910 Posted March 1, 2014 Share Posted March 1, 2014 Depending on the amount of data you have, it may be far easier to just use a portable backup server; and carry the server off-site. There are a couple of nice portable mini-ITX cases that can hold 6 drives. With 4TB drives, this would give you 20TB of protected storage ... or 25TB if you wait for the forthcoming 5TB units that both WD and Seagate are expected to ship "any day now" (at least sometime this year). A couple of nice cases for this ... http://www.newegg.com/Product/Product.aspx?Item=N82E16811345027 http://www.newegg.com/Product/Product.aspx?Item=N82E16811112348 This is what I do. I had an old computer with a handle that was my first unRaid server in this: http://www.newegg.ca/Product/Product.aspx?Item=N82E16811154094 I can get 9 disks into this, and my 24 bay server at present only has 8 disks that need backed up off site. So each week I hook this up and let it run the following script for a few hours. (It is surprisingly how efficient the following script is. About 10 min per 3tb disk if there is no changes) echo "Starting Sync to Tower3" # Set up mounts to Tower3 mkdir /mnt/t3disk1 mkdir /mnt/t3disk2 mkdir /mnt/t3disk3 mkdir /mnt/t3disk4 mkdir /mnt/t3disk5 mkdir /mnt/t3disk6 mkdir /mnt/t3disk7 mkdir /mnt/t3disk8 mkdir /mnt/t3disk9 mount -t nfs tower3:/mnt/disk1/ /mnt/t3disk1 mount -t nfs tower3:/mnt/disk2/ /mnt/t3disk2 mount -t nfs tower3:/mnt/disk3/ /mnt/t3disk3 mount -t nfs tower3:/mnt/disk4/ /mnt/t3disk4 mount -t nfs tower3:/mnt/disk5/ /mnt/t3disk5 mount -t nfs tower3:/mnt/disk6/ /mnt/t3disk6 mount -t nfs tower3:/mnt/disk7/ /mnt/t3disk7 mount -t nfs tower3:/mnt/disk8/ /mnt/t3disk8 mount -t nfs tower3:/mnt/disk9/ /mnt/t3disk9 echo mounts to t3 set up # Backup from production server to 2nd level backup via cron and log results # version .12 # Set up email header echo To: [email protected] >> /boot/logs/cronlogs/t3summary.log echo From: [email protected] >> /boot/logs/cronlogs/t3summary.log echo Subject: Tower3 rsync summary >> /boot/logs/cronlogs/t3summary.log echo >> /boot/logs/cronlogs/t3summary.log # Backup Disk 1 - 9 echo ====== moving to Disk1 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk1/ /mnt/t3disk1/ >> /boot/logs/cronlogs/t3disk1.log echo ====== moving to Disk2 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk2/ /mnt/t3disk2/ >> /boot/logs/cronlogs/t3disk2.log echo ====== moving to Disk3 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk3/ /mnt/t3disk3/ >> /boot/logs/cronlogs/t3disk3.log echo ====== moving to Disk4 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk4/ /mnt/t3disk4/ >> /boot/logs/cronlogs/t3disk4.log echo ====== moving to Disk5 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk5/ /mnt/t3disk5/ >> /boot/logs/cronlogs/t3disk5.log echo ====== moving to Disk6 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk6/ /mnt/t3disk6/ >> /boot/logs/cronlogs/t3disk6.log echo ====== moving to Disk7 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk7/ /mnt/t3disk7/ >> /boot/logs/cronlogs/t3disk7.log echo ====== moving to Disk8 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk8/ /mnt/t3disk8/ >> /boot/logs/cronlogs/t3disk8.log echo ====== moving to Disk9 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log rsync -av --stats --progress /mnt/disk9/ /mnt/t3disk9/ >> /boot/logs/cronlogs/t3disk9.log echo ====== moving to end =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log # Wait for 5 minutes sleep 300 # Release mounts to Tower3 umount /mnt/t3disk1 umount /mnt/t3disk2 umount /mnt/t3disk3 umount /mnt/t3disk4 umount /mnt/t3disk5 umount /mnt/t3disk6 umount /mnt/t3disk7 umount /mnt/t3disk8 umount /mnt/t3disk9 sleep 60 rmdir /mnt/t3disk1 rmdir /mnt/t3disk2 rmdir /mnt/t3disk3 rmdir /mnt/t3disk4 rmdir /mnt/t3disk5 rmdir /mnt/t3disk6 rmdir /mnt/t3disk7 rmdir /mnt/t3disk8 rmdir /mnt/t3disk9 echo mounts to t3 taken down # Send email of summary of results ssmtp [email protected] < /boot/logs/cronlogs/t3summary.log cd /boot/logs/cronlogs mv t3summary.log "`date +%Y%m%d_%H%M`_t3summary.logarchive" When this is done, the servers are mirrored up and the computer with a handle gets carried back offsite. With 9 disks, the backup server weighs about 30 lb. Note: you need to match disk capacities up exactly if you have different sized disks in your system. Disk1 must be the same size on both computers, and Disk2 etc.... Quote Link to comment
garycase Posted March 1, 2014 Share Posted March 1, 2014 Note: you need to match disk capacities up exactly if you have different sized disks in your system. Disk1 must be the same size on both computers, and Disk2 etc.... That's not a requirement unless your goal is to mirror the specific disks on the array (as you're doing). If you simply backup the shares, it doesn't matter whether the disk sizes are the same, or even if the number of disks is the same. I do this with a backup server that has a significantly different disk complement than my main media server. I simply run a Syncback profile (from a Windows machine) that backs up \\Media\DVDs to \\XtraBakups\DVDs [it also does a few other shares, but that's the major share, and you get the idea.] Quote Link to comment
tucansam Posted March 7, 2014 Share Posted March 7, 2014 My $0.02. I decided that, while a major PITA, I can theoretically re-encode all of my DVDs and BluRays should my server blow up. Plus they take up, by far, the most room on the server (and in boxes in the garage, ha!). So my backups are only of the family's critical files, homework, legal papers, taxes, family photos, etc. Stuff that is literally not replaceable. I have two unraid servers. One is full of 4TB drives, a major investment for me. My second server is full of older disks, and when I upgrade in my main server, the second server gets the old drive (ie, I started out with 120 and 320GB drives and now have a few 3TB drives to add when it comes time). I use rsync to copy the critical stuff over to the second server. It has just one share, simply, "backup." I can bring it online in a pinch if the main server is dead and I need data *right now*, or I can use it to copy stuff back. rsync was a major PITA for me, and I have been unable to automate it; it requires near constant babysitting from me, and likes to give me errors and trouble. I don't have a lot of free time anymore, so maybe some day I'll be able to get it all working. For now, once a week (or sooner if something major is added to the main server, like a gig of wedding photos or something), I run a bunch of rsync commands to get the data moved. I like the idea of moving data with Windows, except that, in my experience, Windows copies everything to itself first, and then the second machine, acting like a conduit for the data. So, if I copy/paste from server1 to server2 using Explorer, the process slows because my Windows system is currently on a powerline ethernet adapter. rsync lets the two servers, sitting side-by-side, copy at gig speeds. Once I get my house wired, this won't be so much of an issue, but for now... As for off-site, I would love to be able to do that. I have toyed with the idea of setting up a server at my folks' house, so they can have the benefits of unraid, and each household can backup to the other's server. But even with an initial rsync copy, continuous weekly copying of even just differential data is going to eat up the "allowed bandwidth" our retarded ISPs allow. I'd be getting over quota emails constantly, by the time the kids have watched a few Netflix streams and I've downloaded a bunch of new ISOs. So off-site remains elusive for me, but my eventual goal. My final idea, as has been mentioned, is to simply carry the server to another location (my locker at work for example). This was the idea when I built the second server using an ammo can. But I've yet to actually do it. Yep, my fault. Quote Link to comment
DaleWilliams Posted March 7, 2014 Share Posted March 7, 2014 This was the idea when I built the second server using an ammo can. Quote Link to comment
tucansam Posted March 7, 2014 Share Posted March 7, 2014 This was the idea when I built the second server using an ammo can. Yep. Works fantastic. Quote Link to comment
garycase Posted March 7, 2014 Share Posted March 7, 2014 I looked at your pictures -- the ammo can server is indeed unique Quote Link to comment
Falc410 Posted March 17, 2014 Share Posted March 17, 2014 I'm planning to build 2 servers for my dad, although I don't have much experience with unRaid yet as I have only been reading forums so far. I'm planning to build him a small 6-8TB machine with needs to be backed up occasionally. The Backup server will only be switched on during backup and otherwise stays off. I was also thinking of using unRaid for this server but can I use rsync to backup a complete Share (which might be more than 4TB) to another share on another server (that is automatically distributed over 2 disks)? Or do I need to backup the harddrives individually (like in the script on the 1st page shown in this thread). Than I couldn't quite mix and match disks anymore. Also, do I need a parity disk in the backup machine? Since I don't want too spend too much money, I was hoping to to not have one. Or maybe I should just use some other OS which offers me the ability to configure the HDDs as JBOD. Quote Link to comment
garycase Posted March 17, 2014 Share Posted March 17, 2014 ... do I need a parity disk in the backup machine? Since I don't want too spend too much money, I was hoping to to not have one. There's no requirement that you assign a parity disk. Of course if you don't use one, then the backup system won't be fault-tolerant. On the other hand, writes to it will be much faster, since maintaining the integrity of parity is why writes are relatively slow on UnRAID (since each write requires 4 disk operations). ... Or maybe I should just use some other OS which offers me the ability to configure the HDDs as JBOD. UnRAID without a parity disk is essentially that. You can create a single share (e.g. "MyBigDisk") that uses all of the disks for its storage space. Quote Link to comment
superloopy Posted March 17, 2014 Share Posted March 17, 2014 Why wouldn't you 'parity' a backup server? Surely if the need ever arose for you to recreate those backups you might not be able to do so, hardly a worthy backup in my opinion. We have all been there ..... spend the dosh and don't cut corners. UnRaid must be about the cheapest way to maintain a viable backup. Quote Link to comment
Falc410 Posted March 17, 2014 Share Posted March 17, 2014 Thanks for the answer. I might include a parity disk at a later time, but since the backup system should only run like once in a week (the data is not changed that often, so maybe even bi-weekly) there should be little usage of the harddrives. It's highly unlikely that they will fail right away when I need to do a restore. It's always a cost vs risk tradeoff. If I had more money I would maybe even build a 3rd unRaid server...but I just calculated the total cost and I'm already way over 1000 Euros.... Quote Link to comment
superloopy Posted March 17, 2014 Share Posted March 17, 2014 It's highly unlikely that they will fail right away when I need to do a restore ..... But they WILL fail and you cannot predict in what circumstance .... every drive does, and when you need it, it won't be there. That's when a parity drive will be a godsend. Think on my friend. Quote Link to comment
garycase Posted March 18, 2014 Share Posted March 18, 2014 I certainly agree a parity drive is a good idea. On the other hand, it's also true that the likelihood of 3 simultaneous drive failures -- which is what would be needed to lose data [2 failures on the main UnRAID server and one failure on the backup server] is relatively low Assuming, of course, that you "train" your dad to pay attention to the status of the main server (or install an e-mail notification add-on so you will KNOW when a drive has failed) AND that you pay attention to the status of the backup server as well (again, an e-mail notification would be best). Quote Link to comment
superloopy1 Posted November 14, 2019 Share Posted November 14, 2019 I know this an old thread but i've read it all and think i understand around 75% of it. What i'm after is the exact rsync command which will allow me to sync a SHARE, not a disk, between two servers. Once armed with this i'm sure i can take the script outlined here and give it a whirl. I have two unRaid servers, the backup being around 50TB of unused space after i've upgraded the primary. Seems sense to copy across on a regular basis. Does rsync just copy many changes, allowing the time taken to sync to decrease after the initial sync up? Maybe i'll drop my understanding level to 60% after that question eh! Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.