How do you back up your array?

[RockDawg]

I've long considered building a second unraid server to backup my main unraid server but have never quite got around to it.  Lately that's been troubling me and I'm feeling that I should probably do something sooner rather than later.  So I am again thinking about building a second unraid server to simply be a backup.

 

My question is how many of you do this and how do you handle it?  It seems simple enough on the surface but as I think about it some questions arise:

 

• How did you perform the initial data tansfer? And did you verify the accuracy of the transfer somehow?
• How did the initial transfer affect the performance of the main server while ongoing?
• Does you backup server stay asleep and wake for the backup?  If so, is that automated?
• What about parity checks?  How often and does the server automatically wake for that too?

 

Ideally I would like a "set it and forget it" set up that notifies me after task completion.  Do any of you use and entirely different solution?  If so, what?

Edited August 9, 2020 by RockDawg

[Hoopster]

1 hour ago, RockDawg said:

• How did you perform the initial data tansfer? And did you verify the accuracy of the transfer somehow?
• How did the initial transfer affect the performance of the main server while ongoing?
• Does you backup server stay asleep and wake for the backup?  If so, is that automated?
• What about parity checks?  How often and does the server automatically wake for that too?

• Rsync - yes, with the right flags,  it can  verify the files reach the destination and are an exact copy of the source files
• With the transfer running, I was still able to use the source server.  I really did not see too much of an impact
• My backup server has an IPMI motherboard.  The rsync backup script powers backup server on before running the backup and powers it off when the backup is complete. Before the IPMI motherboard, the script just woke up the server from S3 sleep and put it back to sleep when the backup ended
• Parity checks are run manually in my case, but it could be automated I assume through user scripts

My backup script backs up only new files or those that have been modified.  Once every three months or so another script runs to delete files from the backup server that are no longer on the source server.  This is plenty of time to figure out if something important is missing from the source server.

 

The whole process is documented in this thread.

 

It is set-it-and forget it and has been running automatically with no user intervention at all for over two years.

 

The script does the following:

 

1. power on backup server
2. prepare log files and email information
3. perform a share by share backup
4. write to logs for each share
5. email backup summary to me when complete
6. power off backup server

The script is automated to run once a week through User Scripts.

 

 

[Aran]

I have a second unraid server only to store a backup of my primary unraid server and home pc's. The automated procedure goes like this:

 

cron: 0 2 * * *     - Primary server wakes up the backup server using a scheduled script with the 'etherwake' command.

                          - When backup server is awake it auto-mounts remote smb-shares from primary server.

cron: 15 2 * * *    - a simple script runs cp -Ru command for the user share.

cron: 45 2 * * *   - a simple script runs cp -Ru command for the media share.

cron: 15 3 * * *    - backup server shut down via /sbin/poweroff script.

 

For me this works very well but i only have 4TB of data. I'm using consumer hardware and don't have IPMI. I you have more important data, and a larger array, you may want some more feedback from your backup then @Hoopster method is a good option.

 

 

[RockDawg]

That is exactly how I am looking to achieve it, Hoopster.  Thanks.  I was hoping to hear more feedback.  Really curious to know if many people backup their entire array.  I've been running unraid for ~14 years now and have never had a backup server.  Problem is the larger it grows the more expensive it is to build the backup server.  I have ~86TB on my server now and that works out to be $1800 in hard drives alone that I would need to purchase.  And the longer I put it off the larger that number becomes.

[Aran]

Cloud storage is not an option? (assuming your data plan allows it) I mean, 1800 bucks gives you unlimited cloud storage for +-10 years? Without having to worry about swapping out hard drives.

[RockDawg]

I only have 25 Mbps upload speeds.  It would take about a year to upload everything and that's not leaving any bandwidth for other stuff.