August 18, 201312 yr I'm planning to probably begin a slow backup of my Unraid server by transferring information to various attached 2TB drives. I'll have hooked the drives upside the machine or via a USB dock. But my question is what would be the best filesystem to use for the backups? Do I connect the 2TB to Windows machine and ntfs? Or go ext3 or something else?
August 18, 201312 yr Agreed. This is why I keep hoping for dual array support, but with a separate server you could store them separately giving you some environmental protection (fire, theft. Etc). I'm current considering a Xen/VMware install so that I can run 2 unraids in the same box for this very reason.
August 18, 201312 yr Agreed. This is why I keep hoping for dual array support, but with a separate server you could store them separately giving you some environmental protection (fire, theft. Etc). I'm current considering a Xen/VMware install so that I can run 2 unraids in the same box for this very reason. Also dual arrays do not protect you from problems that may stem from power supply faults, or issues with other shared hardware or software. Dual arrays or two unRAID session in one box can have their uses, I am sure, but backup would not be one of them in my view.
August 18, 201312 yr Author I'm already using VMWare ESXi in with a pass through raid controller for Unraid. But I've been thinking about a change/backup to FreeNas and the ZFS file system. I have a Norco 4020 case and have the room for expansion/backup and may get a 2nd controller and and split the storage within the case.. But I could even hook up drives to a VM Win 7 machine inside the same server/box and back up to that. I really hope to back up thw drives and than pull and store the drives off site. so I'm unsure if unraid or something else is the right approach or not.
August 23, 201312 yr I'm already using VMWare ESXi in with a pass through raid controller for Unraid. But I've been thinking about a change/backup to FreeNas and the ZFS file system. I have a Norco 4020 case and have the room for expansion/backup and may get a 2nd controller and and split the storage within the case.. I am using the simple approach. Use an old machine with a suitcase handle on top for easy portability, get a 2nd copy of unRaid Pro, backup from my Norco 4224 only the disks that a worthy of getting this kind of concern. Here is the script that does it. Wakes up the destination unRaid server and backs up to it, then shuts it down via IPMI, emailing a summary of the backup process. If you don't have IPMI on the destination server, just delete that part. The rest of this works with any unRaid to unRaid over local Gigabit Lan. Typically takes about 10 minutes per disk provided they are in sync. Then grab old server by suitcase handle and lug it offsite.... Rinse and repeat as necessary... (My TamSolutions server referred to in this script is too heavy to take offsite easily, but I have 2 other older machines that this works with fine too.) #define USER_SCRIPT_LABEL Sync to Server1 email results and shutdown #define USER_SCRIPT_DESCR Start the server by IPMI first (works with TamSolutions servers). echo "Starting Sync to Tower3" # Start up Tower3 via ipmi ipmitool -I lan -H 192.168.1.149 -U ADMIN -P ADMIN chassis power on # Wait for 5 minutes echo "waiting for Tower3 to come online..." sleep 300 # Check if host online Tower3 is 114 HOSTS="192.168.1.114" for myHost in $HOSTS do ping -c 1 "$myHost" > /dev/null if [ "$?" -eq 0 ] ; then echo "Hosts up" #============================================================================== # Set up mounts to Tower3 mkdir /mnt/t3disk1 mkdir /mnt/t3disk2 mkdir /mnt/t3disk3 mkdir /mnt/t3disk4 mkdir /mnt/t3disk5 mkdir /mnt/t3disk6 mkdir /mnt/t3disk7 mkdir /mnt/t3disk8 mkdir /mnt/t3disk9 mount -t nfs tower3:/mnt/disk1/ /mnt/t3disk1 mount -t nfs tower3:/mnt/disk2/ /mnt/t3disk2 mount -t nfs tower3:/mnt/disk3/ /mnt/t3disk3 mount -t nfs tower3:/mnt/disk4/ /mnt/t3disk4 mount -t nfs tower3:/mnt/disk5/ /mnt/t3disk5 mount -t nfs tower3:/mnt/disk6/ /mnt/t3disk6 mount -t nfs tower3:/mnt/disk7/ /mnt/t3disk7 mount -t nfs tower3:/mnt/disk8/ /mnt/t3disk8 mount -t nfs tower3:/mnt/disk9/ /mnt/t3disk9 echo mounts to t3 set up # Backup from production server to 2nd level backup via cron and log results # version .12 # Set up email header echo To: [email protected] >> /boot/logs/cronlogs/t3summary.log echo From: [email protected] >> /boot/logs/cronlogs/t3summary.log echo Subject: Tower3 rsync summary >> /boot/logs/cronlogs/t3summary.log echo >> /boot/logs/cronlogs/t3summary.log # Backup Disk 1 echo ++++++ >> /boot/logs/cronlogs/t3disk1.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk1.log echo ++++++ >> /boot/logs/cronlogs/t3disk1.log date echo ====== moving to Disk1 =========== echo ====== moving to Disk1 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk1.log rsync -av --stats --progress /mnt/disk1/ /mnt/t3disk1/ >> /boot/logs/cronlogs/t3disk1.log date >> /boot/logs/cronlogs/t3disk1.log # Backup Disk 2 echo ++++++ >> /boot/logs/cronlogs/t3disk2.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk2.log echo ++++++ >> /boot/logs/cronlogs/t3disk2.log date echo ====== moving to Disk2 =========== echo ====== moving to disk2 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk2.log rsync -av --stats --progress /mnt/disk2/ /mnt/t3disk2/ >> /boot/logs/cronlogs/t3disk2.log date >> /boot/logs/cronlogs/t3disk2.log # Backup Disk 3 echo ++++++ >> /boot/logs/cronlogs/t3disk3.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk3.log echo ++++++ >> /boot/logs/cronlogs/t3disk3.log date echo ====== moving to Disk3 =========== echo ====== moving to disk3 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk3.log rsync -av --stats --progress /mnt/disk3/ /mnt/t3disk3/ >> /boot/logs/cronlogs/t3disk3.log date >> /boot/logs/cronlogs/t3disk3.log # Backup Disk 4 echo ++++++ >> /boot/logs/cronlogs/t3disk4.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk4.log echo ++++++ >> /boot/logs/cronlogs/t3disk4.log date echo ====== moving to Disk4 =========== echo ====== moving to disk4 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk4.log rsync -av --stats --progress /mnt/disk4/ /mnt/t3disk4/ >> /boot/logs/cronlogs/t3disk4.log date >> /boot/logs/cronlogs/t3disk4.log # Backup Disk 5 echo ++++++ >> /boot/logs/cronlogs/t3disk5.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk5.log echo ++++++ >> /boot/logs/cronlogs/t3disk5.log date echo ====== moving to Disk5 =========== echo ====== moving to disk5 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk5.log rsync -av --stats --progress /mnt/disk5/ /mnt/t3disk5/ >> /boot/logs/cronlogs/t3disk5.log date >> /boot/logs/cronlogs/t3disk5.log # Backup Disk 6 echo ++++++ >> /boot/logs/cronlogs/t3disk6.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk6.log echo ++++++ >> /boot/logs/cronlogs/t3disk6.log date echo ====== moving to disk6 =========== echo ====== moving to disk6 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk6.log rsync -av --stats --progress /mnt/disk6/ /mnt/t3disk6/ >> /boot/logs/cronlogs/t3disk6.log date >> /boot/logs/cronlogs/t3disk6.log # Backup Disk 7 echo ++++++ >> /boot/logs/cronlogs/t3disk7.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk7.log echo ++++++ >> /boot/logs/cronlogs/t3disk7.log date echo ====== moving to disk7 =========== echo ====== moving to disk7 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk7.log rsync -av --stats --progress /mnt/disk7/ /mnt/t3disk7/ >> /boot/logs/cronlogs/t3disk7.log date >> /boot/logs/cronlogs/t3disk7.log # Backup Disk 8 echo ++++++ >> /boot/logs/cronlogs/t3disk8.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk8.log echo ++++++ >> /boot/logs/cronlogs/t3disk8.log date echo ====== moving to disk8 =========== echo ====== moving to disk8 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk8.log rsync -av --stats --progress /mnt/disk8/ /mnt/t3disk8/ >> /boot/logs/cronlogs/t3disk8.log date >> /boot/logs/cronlogs/t3disk8.log # Backup Disk 9 echo ++++++ >> /boot/logs/cronlogs/t3disk9.log echo ++++++ starting daily cron now ++++++++++ >> /boot/logs/cronlogs/t3disk9.log echo ++++++ >> /boot/logs/cronlogs/t3disk9.log date echo ====== moving to disk9 =========== echo ====== moving to disk9 =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk9.log rsync -av --stats --progress /mnt/disk9/ /mnt/t3disk9/ >> /boot/logs/cronlogs/t3disk9.log echo ====== moving to end =========== >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3disk9.log date >> /boot/logs/cronlogs/t3summary.log # Send email of summary of results ssmtp [email protected] < /boot/logs/cronlogs/t3summary.log cd /boot/logs/cronlogs mv t3summary.log "`date +%Y%m%d_%H%M`_t3summary.logarchive" # Wait for 5 minutes sleep 300 # Release mounts to Tower3 umount /mnt/t3disk1 umount /mnt/t3disk2 umount /mnt/t3disk3 umount /mnt/t3disk4 umount /mnt/t3disk5 umount /mnt/t3disk6 umount /mnt/t3disk7 umount /mnt/t3disk8 umount /mnt/t3disk9 sleep 60 rmdir /mnt/t3disk1 rmdir /mnt/t3disk2 rmdir /mnt/t3disk3 rmdir /mnt/t3disk4 rmdir /mnt/t3disk5 rmdir /mnt/t3disk6 rmdir /mnt/t3disk7 rmdir /mnt/t3disk8 rmdir /mnt/t3disk9 echo mounts to t3 taken down # Shut down Tower3 via IPMI using a soft shutdown ipmitool -I lan -H 192.168.1.149 -U ADMIN -P ADMIN chassis power soft else echo To: [email protected] >> /boot/logs/cronlogs/t3summary.log echo From: [email protected] >> /boot/logs/cronlogs/t3summary.log echo Subject: Tower3 failed to start >> /boot/logs/cronlogs/t3summary.log echo >> /boot/logs/cronlogs/t3summary.log echo "Host down 192.168.1.114" echo "Host down 192.168.1.114 " >> /boot/logs/cronlogs/t3summary.log date >> /boot/logs/cronlogs/t3summary.log # Send email of summary of results ssmtp [email protected] < /boot/logs/cronlogs/t3summary.log cd /boot/logs/cronlogs mv t3summary.log "`date +%Y%m%d_%H%M`_t3summary.logarchive" fi done
August 23, 201312 yr A few thoughts ... => Copying all your data to another array in the same box is NOT a good backup !! It would be a nice added layer of redundancy, but provides no protection against hardware failure of the system; theft; fire; etc. => Another UnRAID server is a reasonable approach; but if your data is largely static (as is the case for many UnRAID users with large media collections), you don't need an active system -- just copies of your data. => Your idea of using a set of 2TB (or larger) drives is exactly what I do. The file system's irrelevant, as long as it's supported by whatever you use to do the backups (PC, Mac, etc.). I use NTFS, as I do the backups with a Windows machine; but as I noted it doesn't matter. Starting from scratch (as I gather you're doing) will take a fairly long time; but once you're backed up, it's easy to stay that way if you simply copy everything you're copying to the array to your backup at the same time. I keep a backup disk in a caddy; and whenever I write new data to the array, I also write it to the current backup disk. When the backup disk gets full, I simply save it in a drive box and put a new one in the caddy. This effectively makes backups take NO extra time, since the copy is done at the same time the file is being written to UnRAID. Using 2TB backup disks, I only need to change my backup disk about once/year -- but obviously that will be different for everyone, depending on how much data you write. I store my backup disks in a waterproof, fireproof safe => something you couldn't do, of course, with a 2nd UnRAID system These are VERY convenient for keeping the disks in (I've used them for years): http://www.amazon.com/DriveBox-3851-0000-11-Hard-Disk-Case/dp/B004UALLPE/ref=sr_1_1?ie=UTF8&qid=1377286860&sr=8-1&keywords=WiebeTech
August 23, 201312 yr Garycase: Hmm interesting simple solution, I suppose the copy to backup is done automatically by unRaid? How do you handle deletion of media, and knowledge of the directory structure, eg movement of files to clean up? Alex Sent from my Nexus 10 using Tapatalk 4
August 23, 201312 yr I suppose the copy to backup is done automatically by unRaid? No, I do that manually. All new media is copied to my UnRAID from the same PC, so it's easy to do. How do you handle deletion of media, and knowledge of the directory structure, eg movement of files to clean up? I suppose I should be a bit more thorough in describing my process [i omitted a bit earlier ] My primary UnRAID server (28TB) has several shares -- all but one are exclusively media [Movies BTV, Movies WMC, DVDs, Music] ... and the last one [backups] is a "catch all" that has images of all my other systems, and all of our personal data backups from our various computers. The "Backups" folder is done differently than what I outlined above => it's automatically replicated to my 2nd UnRAID server every night; and I also copy it to a pair of 4TB drives that I store in my safe on a weekly basis. This share easily fits on 2 4TB drives (all the images on one; everything else on the other). As for the media shares (the vast majority of my UnRAID data) => the folder structure is identical to that on UnRAID, so that's not an issue; and as for deletions -- I never delete anything from the backup drives When a backup disk gets full, I save a copy of the directory for that disk in PDF format (so I can easily search for any files I may need to restore); then simply store the disk in my safe. I actually do one more "paranoid" thing with both the array and the backup disks -- ALL of my files have MD5's computed and stored in their folder structure, so I can always confirm the validity of any particular folder by simply right-clicking and selecting "Verify Checksums".
August 23, 201312 yr I actually do one more "paranoid" thing with both the array and the backup disks -- ALL of my files have MD5's computed and stored in their folder structure, so I can always confirm the validity of any particular folder by simply right-clicking and selecting "Verify Checksums". How are you storing the MD5 in the folder structure? Is this native windows software that you are using? weebotech is planning on creating an unRaid integrated system based on this. See here: http://lime-technology.com/forum/index.php?topic=28168.msg256902#msg256902
Archived
This topic is now archived and is closed to further replies.